Hey,
I have trouble solving the following task with VTK:
I have a couple of 2D images I obtain from OpenCV that represent slices of a 3D object. I want to stack these 2D images (in z direction) to reconstruct the 3D structure. I use vtkImageAppend for this task. I succeed with adding data to vtkAppend as input. However, instead of adding all the differing slices to my structure my output just consist of the same image stacked multiple times. For clarification, here ist my code and an example output picture. Please note that I stacked the images in Y-axis here just because it is easier to debug like this. Eventually they should be stacked in Z-axis.
void putInto3D()
{
vtkSmartPointer<vtkImageData> outputVtkImage =
vtkSmartPointer<vtkImageData>::New();
vtkSmartPointer<vtkImageAppend> volume =
vtkSmartPointer<vtkImageAppend>::New();
volume->SetAppendAxis(1);
for (int a = 0; a < count; a++)
{
Mat img = imread(name[a]); //opencv-data
//Convert OpenCV data to vtkImageData
outputVtkImage->ShallowCopy(convertCVMatToVtkImageData(img, true));
//Add the vtkImageData as input
volume->AddInputData(outputVtkImage);
volume->Update();
}
render_2d_slices(volume);
}
int render_2d_slices(vtkSmartPointer<vtkImageAppend> volume)
{
vtkRenderWindowInteractor* iren =
vtkRenderWindowInteractor::New();
// Create a viewer for the image
vtkImageViewer2* viewer =
vtkImageViewer2::New();
viewer->SetSliceOrientationToXY();
viewer->SetInputConnection(volume->GetOutputPort());
viewer->SetSlice(1); //Eventually I want to scroll through slices in Z-direction here
viewer->UpdateDisplayExtent(); //test
viewer->Modified(); //test
viewer->Render(); //test
viewer->SetupInteractor(iren);
// Initialize the event loop and then start it.
iren->Initialize();
iren->Start();
iren->Delete();
viewer->Delete();
return 1;
}
The rendered output looks like this (the program appended the same image 3 times). The photo of a arduino is just an example here.
I can confirm that outputVtkImage contains a different image every time the loop runs through. However, in the end I have the same image (it is always the last image I pass) stacked multiple times instead of the differing images I pass to AddInputData.
Each time you call Update() on vtkImageAppend, it computes its output de novo from the input vtkImageData objects. The problem is that you are re-using the same vtkImageData object for each iteration.
First iteration:
First Input: outputVtkImage
Output: a copy of outputVtkImage
Second iteration:
First Input: outputVtkImage
Second Input: outputVtkImage
Output: outputVtkImage appended with itself
Etcetera. To fix this, you must create a new vtkImageData object for each input. Also, after setting the three inputs, you only need to call Update() once:
void putInto3D()
{
vtkSmartPointer<vtkImageAppend> volume =
vtkSmartPointer<vtkImageAppend>::New();
volume->SetAppendAxis(2);
for (int a = 0; a < count; a++)
{
Mat img = imread(name[a]); //opencv-data
//Convert OpenCV data to vtkImageData
vtkSmartPointer<vtkImageData> outputVtkImage =
vtkSmartPointer<vtkImageData>::New();
outputVtkImage->ShallowCopy(convertCVMatToVtkImageData(img, true));
//Add the vtkImageData as input
volume->AddInputData(outputVtkImage);
}
volume->Update();
render_2d_slices(volume);
}
Thank you so much, that was a very quick solution for a problem that plagued me for days.
Just so I understand it: I was under the impression that because convertCVMatToVtkImageData changes every iteration and then the adresses to it get copied into outputVtkImage the latter would also change every iteration. Is that not how ShallowCopy works?
Now that appending worked I want to render the stack in 3D. Please tell me when I should open a new topic for this. It would be awesome if you or someone else can help me on that, too. (Note that I only set the opacity in order to see if the stack rendering worked correctly).
int render_3d(vtkSmartPointer<vtkImageAppend> volume)
{
// Create a mapper and actor
vtkSmartPointer<vtkDataSetMapper> mapper =
vtkSmartPointer<vtkDataSetMapper>::New();
mapper->SetInputConnection(volume->GetOutputPort());
vtkSmartPointer<vtkActor> actor =
vtkSmartPointer<vtkActor>::New();
actor->SetMapper(mapper);
// Renderer and render window
vtkSmartPointer<vtkRenderer> renderer =
vtkSmartPointer<vtkRenderer>::New();
vtkSmartPointer<vtkRenderWindow> renderWindow =
vtkSmartPointer<vtkRenderWindow>::New();
renderWindow->AddRenderer(renderer);
renderer->AddActor(actor);
actor->GetProperty()->SetOpacity(0.5);
// An interactor
vtkSmartPointer<vtkRenderWindowInteractor> renderWindowInteractor =
vtkSmartPointer<vtkRenderWindowInteractor>::New();
renderWindowInteractor->SetRenderWindow(renderWindow);
vtkSmartPointer<vtkInteractorStyleTrackballCamera> style =
vtkSmartPointer<vtkInteractorStyleTrackballCamera>::New();
renderWindowInteractor->SetInteractorStyle(style);
// Render
renderWindow->Render();
renderWindowInteractor->Initialize();
renderWindow->Render();
// Begin mouse interaction
renderWindowInteractor->Start();
return EXIT_SUCCESS;
}
Using render_2d_slices I can see that ‘volume’ indeed has all 6 slices appended. However when I render it in 3D I can indeed see the stacked slices. However, there are only the first (the one with the ‘0’ on it) and last slice (‘6’). Again, note that I set an opacity so that all slices can be seen here.
I am not sure whether I am rendering it with the wrong functions or if there is some error in my code.
Again, thank you very much for your help.
Yes, that’s how ShallowCopy works. I think your misunderstanding is with how AddInputData() and Update() work. When you call AddInputData(), it stores the address of the container (in this case, the address of the outputVtkImage object), not the address of the contents of the container (which is what ShallowCopy() copies).
The reason you only see the first and last slice is that vtkActor renders polygons: the vtkDataSetMapper converts your image volume into polygons by extracting its surface and converting the surface to polygons. Hence, only the slices on the surface are visible.
If you want to display your volume with proper transparency, the volume mappers are probably the best way (but I’ve never used the volume mappers with RGB data, so I’m not sure which one is the best to use).
