Wrong behaviour stacking different images with vtkImageAppend

Hey,
I have trouble solving the following task with VTK:
I have a couple of 2D images I obtain from OpenCV that represent slices of a 3D object. I want to stack these 2D images (in z direction) to reconstruct the 3D structure. I use vtkImageAppend for this task. I succeed with adding data to vtkAppend as input. However, instead of adding all the differing slices to my structure my output just consist of the same image stacked multiple times. For clarification, here ist my code and an example output picture. Please note that I stacked the images in Y-axis here just because it is easier to debug like this. Eventually they should be stacked in Z-axis.

void putInto3D()
{
    vtkSmartPointer<vtkImageData> outputVtkImage = 
	    vtkSmartPointer<vtkImageData>::New();

    vtkSmartPointer<vtkImageAppend> volume =
	vtkSmartPointer<vtkImageAppend>::New();

    volume->SetAppendAxis(1);

    for (int a = 0; a < count; a++)
    {
	Mat img = imread(name[a]); //opencv-data
	
            //Convert OpenCV data to vtkImageData
	outputVtkImage->ShallowCopy(convertCVMatToVtkImageData(img, true)); 

            //Add the vtkImageData as input
	volume->AddInputData(outputVtkImage);
	volume->Update();
    }
    render_2d_slices(volume);
}

int render_2d_slices(vtkSmartPointer<vtkImageAppend> volume)
{
    vtkRenderWindowInteractor* iren =
	vtkRenderWindowInteractor::New();

// Create a viewer for the image
vtkImageViewer2* viewer = 
	vtkImageViewer2::New();

viewer->SetSliceOrientationToXY();
viewer->SetInputConnection(volume->GetOutputPort());
viewer->SetSlice(1); //Eventually I want to scroll through slices in Z-direction here

viewer->UpdateDisplayExtent(); //test
viewer->Modified(); //test
viewer->Render(); //test

viewer->SetupInteractor(iren);

// Initialize the event loop and then start it.
iren->Initialize();
iren->Start();

iren->Delete();
viewer->Delete();

return 1;
}

The rendered output looks like this (the program appended the same image 3 times). The photo of a arduino is just an example here.

I can confirm that outputVtkImage contains a different image every time the loop runs through. However, in the end I have the same image (it is always the last image I pass) stacked multiple times instead of the differing images I pass to AddInputData.

Any help on this is appreciated. Thank you.

Each time you call Update() on vtkImageAppend, it computes its output de novo from the input vtkImageData objects. The problem is that you are re-using the same vtkImageData object for each iteration.

First iteration:

  • First Input: outputVtkImage
  • Output: a copy of outputVtkImage

Second iteration:

  • First Input: outputVtkImage
  • Second Input: outputVtkImage
  • Output: outputVtkImage appended with itself

Etcetera. To fix this, you must create a new vtkImageData object for each input. Also, after setting the three inputs, you only need to call Update() once:

void putInto3D()
{
    vtkSmartPointer<vtkImageAppend> volume =
        vtkSmartPointer<vtkImageAppend>::New();

    volume->SetAppendAxis(2);

    for (int a = 0; a < count; a++)
    {
        Mat img = imread(name[a]); //opencv-data
	
            //Convert OpenCV data to vtkImageData
        vtkSmartPointer<vtkImageData> outputVtkImage = 
            vtkSmartPointer<vtkImageData>::New();
        outputVtkImage->ShallowCopy(convertCVMatToVtkImageData(img, true)); 

            //Add the vtkImageData as input
        volume->AddInputData(outputVtkImage);
    }

    volume->Update();
    render_2d_slices(volume);
}
1 Like

Thank you so much, that was a very quick solution for a problem that plagued me for days.
Just so I understand it: I was under the impression that because convertCVMatToVtkImageData changes every iteration and then the adresses to it get copied into outputVtkImage the latter would also change every iteration. Is that not how ShallowCopy works?

Now that appending worked I want to render the stack in 3D. Please tell me when I should open a new topic for this. It would be awesome if you or someone else can help me on that, too. (Note that I only set the opacity in order to see if the stack rendering worked correctly).

int render_3d(vtkSmartPointer<vtkImageAppend> volume)
{
    // Create a mapper and actor
    vtkSmartPointer<vtkDataSetMapper> mapper =
        vtkSmartPointer<vtkDataSetMapper>::New();
    mapper->SetInputConnection(volume->GetOutputPort());

    vtkSmartPointer<vtkActor> actor =
        vtkSmartPointer<vtkActor>::New();
    actor->SetMapper(mapper);

    // Renderer and render window
    vtkSmartPointer<vtkRenderer> renderer =
        vtkSmartPointer<vtkRenderer>::New();
    vtkSmartPointer<vtkRenderWindow> renderWindow =
        vtkSmartPointer<vtkRenderWindow>::New();
    renderWindow->AddRenderer(renderer);
    renderer->AddActor(actor);
    actor->GetProperty()->SetOpacity(0.5);

    // An interactor
    vtkSmartPointer<vtkRenderWindowInteractor> renderWindowInteractor =
        vtkSmartPointer<vtkRenderWindowInteractor>::New();
    renderWindowInteractor->SetRenderWindow(renderWindow);

    vtkSmartPointer<vtkInteractorStyleTrackballCamera> style =
        vtkSmartPointer<vtkInteractorStyleTrackballCamera>::New();

    renderWindowInteractor->SetInteractorStyle(style);

    // Render
    renderWindow->Render();

    renderWindowInteractor->Initialize();
    renderWindow->Render();

    // Begin mouse interaction
    renderWindowInteractor->Start();

    return EXIT_SUCCESS;
}

Using render_2d_slices I can see that ‘volume’ indeed has all 6 slices appended. However when I render it in 3D I can indeed see the stacked slices. However, there are only the first (the one with the ‘0’ on it) and last slice (‘6’). Again, note that I set an opacity so that all slices can be seen here.
PNG ardu3d_t
I am not sure whether I am rendering it with the wrong functions or if there is some error in my code.
Again, thank you very much for your help.

Yes, that’s how ShallowCopy works. I think your misunderstanding is with how AddInputData() and Update() work. When you call AddInputData(), it stores the address of the container (in this case, the address of the outputVtkImage object), not the address of the contents of the container (which is what ShallowCopy() copies).

The reason you only see the first and last slice is that vtkActor renders polygons: the vtkDataSetMapper converts your image volume into polygons by extracting its surface and converting the surface to polygons. Hence, only the slices on the surface are visible.

If you want to display your volume with proper transparency, the volume mappers are probably the best way (but I’ve never used the volume mappers with RGB data, so I’m not sure which one is the best to use).

That made it very clear, thanks a lot!
Also, thanks for the help regarding the rendering. I will look into the volume mappers then.