I want to use vtk for registration visualization

Support
1 
#include<vtkSmartPointer.h>
#include<vtkDataObject.h>
#include<vtkImageFlip.h>
#include<vtkImageViewer2.h>
#include<vtkRenderWindow.h>
#include <vtkAutoInit.h>
#include<vtkImageData.h>
#include<vtkImageMapper.h>
#include<vtkImageActor.h>
#include<vtkRenderer.h>
#include<vtkRenderWindowInteractor.h>
#include<string>
#include <thread>
#include<Windows.h>
#include<functional>
VTK_MODULE_INIT(vtkRenderingOpenGL2);
VTK_MODULE_INIT(vtkInteractionStyle);
VTK_MODULE_INIT(vtkRenderingFreeType);
class VisualRegistration
{
public:
	VisualRegistration()
	{
		actor =vtkSmartPointer<vtkImageActor>::New();
		render =vtkSmartPointer<vtkRenderer>::New();
		window =vtkSmartPointer<vtkRenderWindow>::New();
	}
	~VisualRegistration();
	int i = 0;

	void Updata(vtkImageFlip* image) {
	
	
		actor->SetInputData(image->GetOutput());
		render->AddActor(actor);
		render->ResetCamera();
		render->SetBackground(1, 1, 1);
		window->AddRenderer(render);
		window->SetSize(640, 480);
		window->SetWindowName("ImageViewer3D");
		window->Render();

	}

private:
	vtkSmartPointer<vtkImageActor> actor;
	vtkSmartPointer<vtkRenderer> render;
	vtkSmartPointer<vtkRenderWindow> window;
};

This is my code. I call Updata to update the image and render the window, but the window will not respond randomly, which happens very frequently. Updata is called in the optimizer IterationEvent in the registration.

2

Why do frequent calls to render cause the window to not respond

3

The window doesn’t respond because the application is busy doing the registration and the rendering.

In order to do this properly, you need two things:

  1. The ITK registration must be in a separate thread, because it has to run concurrently with the user interface.
  2. The update and the render must be done with a timer because if they are called too frequently, the user interface will become unresponsive.

In other words, calling Updata() in IterationEvent is the wrong approach. The correct approach is much more complicated.

4

I didn’t add an interactor before, but now I have added an interactor and set up a separate thread for the rendering part, so there will be no no response. But there is a new problem. I can’t call render on the registration thread, which will lead to errors. I looked up the information and found that opengl must be called on the created thread. Can you tell me what the correct and complex method is?

5

I found a example https://examples.itk.org/src/registration/common/watchregistration/documentation it is right ?

6

Only the registration should be in a separate thread.

The rendering should be done in the main thread. You can use the CreateRepeatingTimer() method to create a timer. It will cause the interactor to repeatedly emit vtkCommand::TimerEvent, so you can catch this event with AddObserver() and call Render(). Do not call Render() for IterationEvent. That will not work.

For IterationEvent, I recommend that you copy the ITK image data (from the registration) into a VTK image data, and this should be done with a mutex lock so that the copy does not happen at the same time as a VTK update.

I will not give you all the details, because then this email would be very long.

7

Thank you very much. It’s already very helpful.

8

I want to ask again how to put the registration in a separate thread. When I call update at the end of the registration, I put the update in a separate thread. Is the whole registration in a separate thread?

9

Show me your registration code. Otherwise, all I can do is guess.

If you are using an ITK optimizer, then StartOptimization() must be called in a separate thread. But after optimization is complete, you don’t need that thread anymore.

10

Note that live display of the evolving transform is not a useful feature, because you cannot guide the registration; and generally in a real-world use nobody has time to just sit and watch how an image is getting aligned - you just want to get the end result as soon as possible. Live visualization may even impede the registration process because applying the transformation and render the results may slow things down (especially if you are visualizing a non-linear transform).

However, I acknowledge that this kind of live display can be a cool animation for presentations, and maybe useful for some troubleshooting or research.

A couple of days ago, jiazeyu asked for the same feature on the Slicer forum. Maybe you can team up with him:

Slicer runs the ITK registration (BRAINS, Elastix, or ANTs) in a separate process and it can already display the transformation or transformed images, models, point sets, etc. in real-time. The missing piece is just passing the registration parameters from the registration process to Slicer. It should be all straightforward and requires only very little development (make the registration process display transformation parameters on the standard output of the process or write to file; then have a Python script that observes the output and updates the transform parameters).

12


马文龙marlon.ma

1m

I have simply completed the function through the guidance of dgobbi. I put the rendering on the main thread, and the registration is executed by a sub thread. In the iteration event of the optimizer, only the image is copied to the rendering, and the lock is added. Since a new thread has been opened, I do not know how much the impact on the registration speed is.

13

If you can use all your CPU cores for computing the registration metric but instead you use some of the cores for resampling the image for visualization then the registration will slow down.

Very rough approximation of the slowdown: let’s assume you use 10% of the image voxels for registration. If you display 3 image slices, 1000x1000 pixels each, that results in about 3000000/(256x256x256) samples = 18% of the voxels. Registration time is not just sampling of the image and rendering is not just resampling, so that would add some time to both the registration and visualization, but still, overall, more samples mean more computation time. If you render the result of each registration iteration then you may need to work with 3x more samples, potentially making the entire registration process 3x longer.

Of course, you can improve the performance, for example by showing the result of every 10th registration iteration and you may display images at lower resolution. So, you can probably find some visualization parameters that result in acceptable rendering quality and update rate with a tolerable slowdown.

14

Thank you for your explanation. I understand.