Having difficulties understanding how to control what I am rendering in 2D

Support
1

Hello. I have been trying to do somethign that shoudl be very simple . I am loading a dicom image (ok, done) and render it to a buffer (already did it to a file, and will handle the buffer target later). But I cannot grasp how I set What part of the image I want to render. I have an image that is for example 2048x3096 pixels but I want to render only a 2d slice of it that starts at X,Y and render it to an image of certain dimensions.

All examples I find use iteractors to handle user interaction, but that is not what I want, I already have the parameters of what part of the image I want, but I cannot find to what component in the (Rather complicated let me add, VTK pipeline).

Can anyone point me to where I can find what I need?

2

You can use vtkImageReslice filter to extract any part of an image in any orientation at any resolution.

3

Thank you. Using the ImageReslice I was able to extract the desired part of the image. That said, I am still struggling with doing so at a target resolution (for example I want to get a square segment that is 512x512 from a larger image, but render it into a 256x256 image. I am trying to do so by using the vtkImageResize method SetOutputDimensions(256,256,1) (on the output of the REslice). When I do that although VTK throws an error :

5 ERR| vtkImageResize (0x5573d537f820): ThreadedRequestData: output scalar type does not match input scalar type

I could not find anywhere in the internet this error message alongside vtk Resize. Am I doing somethign very wrong or is it some other step I am missing in the pipeline ? A sample of how I am doing it:

vtkSmartPointer reader =
vtkSmartPointer::New();
reader->SetFileName(inputFilename.c_str());
reader->Update();

// Setup offscreen rendering
vtkSmartPointer<vtkGraphicsFactory> graphics_factory =
        vtkSmartPointer<vtkGraphicsFactory>::New();
graphics_factory->SetOffScreenOnlyMode( 1);
graphics_factory->SetUseMesaClasses( 1 );

auto image_ptr = reader->GetOutput();


vtkSmartPointer<vtkImageReslice> reslice =
        vtkSmartPointer<vtkImageReslice>::New();
reslice->SetOutputExtent(0, 100, 0, 100, 0, 0);
reslice->SetOutputOrigin(150,50,0);
reslice->SetInputData(reader->GetOutput());

vtkSmartPointer<vtkImageResize> resize =
    vtkSmartPointer<vtkImageResize>::New();
resize->SetInputData(reslice->GetOutput());
resize->SetOutputDimensions(256, 256, 1);
  //resize->SetMagnificationFactors(1.0,1.0,1.0);

vtkSmartPointer<vtkImageMapper> mapper = vtkSmartPointer<vtkImageMapper>::New();
mapper->SetInputConnection(resize->GetOutputPort());


vtkSmartPointer<vtkActor2D> actor =
        vtkSmartPointer<vtkActor2D>::New();
actor->SetMapper(mapper);
actor->SetPosition(0,0);

vtkSmartPointer<vtkRenderer> renderer =
        vtkSmartPointer<vtkRenderer>::New();
vtkSmartPointer<vtkRenderWindow> renderWindow =
        vtkSmartPointer<vtkRenderWindow>::New();
renderWindow->SetOffScreenRendering( 1 );
renderWindow->AddRenderer(renderer);


// Add the actors to the scene
renderer->AddActor(actor);

//renderer->SetActiveCamera(camera);


renderWindow->SetSize(512,512);
renderWindow->Render();

vtkSmartPointer<vtkWindowToImageFilter> windowToImageFilter =
        vtkSmartPointer<vtkWindowToImageFilter>::New();

windowToImageFilter->SetInput(renderWindow);
windowToImageFilter->Update();

vtkSmartPointer<vtkPNGWriter> writer =
        vtkSmartPointer<vtkPNGWriter>::New();
writer->SetFileName("screenshot.png");
writer->SetInputConnection(windowToImageFilter->GetOutputPort());
writer->Write();
4

Maybe vtkImageResize expects you to set the output scalar type manually to the same scalar type as the input image.

5

If i take the output in a temporary holder and get the Scalar type it ha the exact same scalar type as the input (scalar value 11). I am getting really confused with this component. Earlier you said that ImageReslice could already achieve the extraction to a desired resolution. Is there some example where I can see how that is done?

6

Yes, you can specify output spacing to the reslice filter and generally it is preferable to do all the resampling in a single step, in vtkImageReslice.

7

Sorry for the long chain of questions, but I am struggling really hard with VTK ( I wrote a whole framework for these stuff in my previous job, but here I need to use VTK and I am probably struggling mostly because the line of thought behind VTK is very different from what I had in my own framework).

When I use the OutputSpacing outputOrigin adn OutputExtension I do get respositioning and sclaing effects, but they do not make sense for me.

I have an image that is 512x512. When I try to use the Origin and extension as 256 what I get is not half of the image as displacement, it is a bit over 3/4 of the image.

I tried to check the values these attributes have before I try to set them, with the GetMethods just after I have set the InputPort as the output of a vtk Image. … but the values are always zero. I am having a really hard time to understand what the parameters of the methods are supposed to be (and if they are in pixels or mm ) since they are named a,b,c,d,e … etc…

If I have an image of 512x512 and I want to get the upper right quadrant of the image, scaled up to 512x512. What should these parameters be? (On my own framework I would just pass a corner and 2 direction vectors in pixel space, and a parameter image where to render to (The zoom woudl be implicit from the image size)… maybe I am getting confused because of some fundamental difference in the way of thinking.

8

Don’t worry, VTK is quite simple to use, you just happen to come across vtkImageReslice, which I find the one of the most powerful but hardest-to-use class in VTK. It can apply inverse non-linear transforms on the fly for extracting a slice at arbitrary location and resolution, using different interpolators, accepting an input stencil and generating output stencil for images arbitrary type and number of scalar components, etc. and doing all these very efficiently, all in a single class. As you would expect, this makes it complicated.

It also has somewhat redundant API (you can achieve the same thing with different combinations of ResliceAxes and ResliceTransform and SetOutputX), which is good for flexibility but requires you to learn a lot about the class before you can use it effectively.

An additional unnecessary complexity is that you cannot set output geometry by specifying a reference image, because support of arbitrary image directions have been just recently added to vtkImageData. If you spend some time with understanding this class anyway, it would be very nice if you could add this convenience feature (then you could just specify an output image and the class would give you the content of the input image there).

vtkImageReslice can do something like this, too, by setting reslice axes origin and directions, and output spacing.

Alternatively, you can specify a reslice transform that contains relative transform between the physical image coordinate system of the input image and the output slice. We use this in 3D Slicer, where the input volume can be under a series of arbitrary transforms.

If you are not sure what’s happening under the hood then you can have a look at the source code.

9

Well, I am still 6 hours of attempts later, baffled. It does not matter the values that I use in ResliceAxisOrigin(Be it zero or any value, it changes absolutely nothing), but when I change the ResliceAxisCosine the extent of the reslice change. Since both only applies values to the internal matrix I am baffled why the origin simply has no effect on the reslicing. To reiterate, I want the reslicing to start at the center of the image and progress only into the upper right quadrant. The origin seems to influence the cneter of rotation as I expected, but it seems it is not taken into account as to where the sampling starts. How I set that is my struggle.

Also something that I must confirm. The reslice axis only imply in a transformation of the image space scale and direction, it does not enforce any limit (so If my axis are (0.5,0,0) (0,0.5,0) and I halve the output Spacing I still shall see the full image. To limit my extraction to half the image regardless of the resolution I want to render I need to use the ExtractVoi class?

10

Ok, finnally I got something working. Instead of using the SetResliceAxisOrigin etc I created a whole new matrix, setup it manually and passed it to the reslice instance and now it seems to be working. That still leaves me confused… to set the reslice Axis and resliceAxisOrigin should not have the same result as to set the Matrix as a whole?

11

You can check such details by reading the source code.

12

@Tiago_D_Agostini if there is any improvement to the documentation that you think would help you and others in the future then please send a merge request with the proposed changes. Thank you!