vtkResliceCursorWidget: output images are cropped

Support
1

Hi to all,

I am trying to set up a four pane medical image viewer in Python, but I notice that the displayed images seem to be cropped, especially when more than one cursor has been rotated. Here you can see a GIF showing my problem in the green pane when the cursors on the other two planes have been rotated:
FourPaneViewer
At first I thought the problem was somehow correlated with this option of vtkImageReslice:

SetAutoCropOutput(): Turn this on if you want to guarantee that the extent of the output will be large enough to ensure that none of the data will be cropped (default: Off).

So I set it on, but it didn’t work.

Here you can find my code:

from vtk import *

# Reslice cursor callback
def ResliceCursorCallback(obj,event):
    for i in range(0,3):
        ps = planeWidgetArray[i].GetPolyDataAlgorithm();
        origin = resliceCursorWidgetArray[i].GetResliceCursorRepresentation().GetPlaneSource().GetOrigin();
        ps.SetOrigin(origin);
        point1 = resliceCursorWidgetArray[i].GetResliceCursorRepresentation().GetPlaneSource().GetPoint1();
        ps.SetPoint1(point1);
        point2 = resliceCursorWidgetArray[i].GetResliceCursorRepresentation().GetPlaneSource().GetPoint2();
        ps.SetPoint2(point2);
        planeWidgetArray[i].UpdatePlacement();

# DICOM reader
reader = vtkDICOMImageReader();
reader.SetDirectoryName("CT");
reader.Update();
imageDims = reader.GetOutput().GetDimensions();

# // Bounding box
outline = vtkOutlineFilter();
outline.SetInputConnection(reader.GetOutputPort());

outlineMapper = vtkPolyDataMapper();
outlineMapper.SetInputConnection(outline.GetOutputPort());

outlineActor = vtkActor();
outlineActor.SetMapper(outlineMapper);
outlineActor.GetProperty().SetColor(1,1,0);

# // Mapper and actors for volume
volumeMapper = vtkPolyDataMapper();
volumeMapper.SetInputConnection(reader.GetOutputPort());

volumeActor = vtkActor();
volumeActor.SetMapper(volumeMapper);

# // Renderers
renWin = vtkRenderWindow();
RendererArray = [None]*4;
for i in range(0,4):
    RendererArray[i] = vtkRenderer();
    renWin.AddRenderer(RendererArray[i]);
renWin.SetMultiSamples(0);

# // Render window interactor
iren = vtkRenderWindowInteractor();
renWin.SetInteractor(iren);

# // Picker
picker = vtkCellPicker();
picker.SetTolerance(0.005);

# // Properties
ipwProp = vtkProperty();

# // 3D plane widgets
planeWidgetArray = [None]*3;
for i in range(0,3):
    planeWidgetArray[i] = vtkImagePlaneWidget();
    planeWidgetArray[i].SetInteractor(iren);
    planeWidgetArray[i].SetPicker(picker);
    planeWidgetArray[i].RestrictPlaneToVolumeOn();
    color = [0, 0, 0 ];
    color[i] = 1;
    planeWidgetArray[i].GetPlaneProperty().SetColor(color);
    planeWidgetArray[i].SetTexturePlaneProperty(ipwProp);
    planeWidgetArray[i].TextureInterpolateOff();
    planeWidgetArray[i].SetResliceInterpolateToLinear();
    planeWidgetArray[i].SetInputConnection(reader.GetOutputPort());
    planeWidgetArray[i].SetPlaneOrientation(i);
    planeWidgetArray[i].SetSliceIndex(int(imageDims[i] / 2));
    planeWidgetArray[i].DisplayTextOn();
    planeWidgetArray[i].SetDefaultRenderer(RendererArray[3]);
    planeWidgetArray[i].SetWindowLevel(1358, -27, 0);
    planeWidgetArray[i].On();
    planeWidgetArray[i].InteractionOff();

planeWidgetArray[1].SetLookupTable(planeWidgetArray[0].GetLookupTable()); 
planeWidgetArray[2].SetLookupTable(planeWidgetArray[0].GetLookupTable());

# // ResliceCursor
resliceCursor = vtkResliceCursor();
center = reader.GetOutput().GetCenter();
resliceCursor.SetCenter(center[0], center[1], center[2]);
resliceCursor.SetThickMode(0);
resliceCursor.SetThickness(10, 10, 10);
resliceCursor.SetHole(0);
resliceCursor.SetImage(reader.GetOutput());

# // 2D Reslice cursor widgets
resliceCursorRepArray = [None]*3;
resliceCursorWidgetArray = [None]*3;
viewUp = [[0, 0, -1],[0, 0, -1],[0, 1, 0]];
for i in range(0,3):
    resliceCursorWidgetArray[i] = vtkResliceCursorWidget();
    resliceCursorRepArray[i] = vtkResliceCursorLineRepresentation();
    resliceCursorWidgetArray[i].SetInteractor(iren);
    resliceCursorWidgetArray[i].SetRepresentation(resliceCursorRepArray[i]);
    resliceCursorRepArray[i].GetResliceCursorActor().GetCursorAlgorithm().SetResliceCursor(resliceCursor);
    resliceCursorRepArray[i].GetResliceCursorActor().GetCursorAlgorithm().SetReslicePlaneNormal(i);
    minVal = reader.GetOutput().GetScalarRange()[0];
    reslice = resliceCursorRepArray[i].GetReslice();
    reslice.SetInputConnection(reader.GetOutputPort());
    reslice.SetBackgroundColor(minVal, minVal, minVal, minVal);
    reslice.AutoCropOutputOn();
    reslice.Update();
    resliceCursorWidgetArray[i].SetDefaultRenderer(RendererArray[i]);
    resliceCursorWidgetArray[i].SetEnabled(1);
    RendererArray[i].GetActiveCamera().SetFocalPoint(0, 0, 0);
    camPos = [0, 0, 0];
    camPos[i] = 1;
    RendererArray[i].GetActiveCamera().SetPosition(camPos[0], camPos[1], camPos[2]);
    RendererArray[i].GetActiveCamera().ParallelProjectionOn();
    RendererArray[i].GetActiveCamera().SetViewUp(viewUp[i][0], viewUp[i][1], viewUp[i][2]);
    RendererArray[i].ResetCamera();
    resliceCursorWidgetArray[i].AddObserver('InteractionEvent',ResliceCursorCallback)
    range_color = reader.GetOutput().GetScalarRange();
    resliceCursorRepArray[i].SetWindowLevel(range_color[1] - range_color[0], (range_color[0] + range_color[1]) / 2.0, 0);
    planeWidgetArray[i].SetWindowLevel(range_color[1] - range_color[0], (range_color[0] + range_color[1]) / 2.0, 0);
    resliceCursorRepArray[i].SetLookupTable(resliceCursorRepArray[0].GetLookupTable());
    planeWidgetArray[i].GetColorMap().SetLookupTable(resliceCursorRepArray[0].GetLookupTable());
    
# // Background
RendererArray[0].SetBackground(0.3, 0.1, 0.1);
RendererArray[1].SetBackground(0.1, 0.3, 0.1);
RendererArray[2].SetBackground(0.1, 0.1, 0.3);
RendererArray[3].AddActor(volumeActor);
RendererArray[3].AddActor(outlineActor);
RendererArray[3].SetBackground(0.1, 0.1, 0.1);
renWin.SetSize(600, 600);

# // Render
RendererArray[0].SetViewport(0, 0, 0.5, 0.5);
RendererArray[1].SetViewport(0.5, 0, 1, 0.5);
RendererArray[2].SetViewport(0, 0.5, 0.5, 1);
RendererArray[3].SetViewport(0.5, 0.5, 1, 1);
renWin.Render();

# // Camera in 3D view
RendererArray[3].GetActiveCamera().Elevation(110);
RendererArray[3].GetActiveCamera().SetViewUp(0, 0, -1);
RendererArray[3].GetActiveCamera().Azimuth(45);
RendererArray[3].GetActiveCamera().Dolly(1.15);
RendererArray[3].ResetCameraClippingRange();

iren.Initialize()
iren.Start()

Has someone any idea about which can be the cause of this weird behaviour?

Thanks in advance for any help or hints.

2 Likes
2

can you resoved it?

3

maybe @lassoan or @LucasGandel can have already encounter this issue and have some insight ?

4

This was the behavior 4 years ago, but it should have been fixed in https://gitlab.kitware.com/vtk/vtk/-/commit/0e0af1b9a9a96efafcf8ce863ff87a3e67b665e3 which is available in vtk 9.1.
@zist8888 do you reproduce with recent versions of VTK?

1 Like
5

yes,output images show correct.but I have an other question,when move navigator lines in near other view,the line also can operate at same direction.I think the line is not show this view,so cant operate.can you give some suggestion,thanks!

6

Do you mean that the line is not always visible? It’s probably related to this issue: https://gitlab.kitware.com/vtk/vtk/-/issues/18441

Using the following properties for the lines should fix it:

    resliceCursorRep[i]->GetResliceCursorActor()->GetCenterlineProperty(0)->SetRepresentationToWireframe();
    resliceCursorRep[i]->GetResliceCursorActor()->GetCenterlineProperty(1)->SetRepresentationToWireframe();
    resliceCursorRep[i]->GetResliceCursorActor()->GetCenterlineProperty(2)->SetRepresentationToWireframe();
7

no,It meas that navigator lines show correct,but the line was operated part is incorrect.because in other view also can operate, it should be operated by own view.

9633e1da2637df0411ac44b2978163f9
9633e1da2637df0411ac44b2978163f9606Ă—633 40.4 KB

8

OK thanks for the additional information. This probably happens because you have multiple renderers but only one render window and one interactor. In this case can you try to set the DefaultRenderer for every widgets to see if it helps? If you could share your code that would help a lot.

10 
from vtk import *

# Reslice cursor callback
def ResliceCursorCallback(obj,event):
    for i in range(0,3):
        ps = planeWidgetArray[i].GetPolyDataAlgorithm();
        origin = resliceCursorWidgetArray[i].GetResliceCursorRepresentation().GetPlaneSource().GetOrigin();
        ps.SetOrigin(origin);
        point1 = resliceCursorWidgetArray[i].GetResliceCursorRepresentation().GetPlaneSource().GetPoint1();
        ps.SetPoint1(point1);
        point2 = resliceCursorWidgetArray[i].GetResliceCursorRepresentation().GetPlaneSource().GetPoint2();
        ps.SetPoint2(point2);
        planeWidgetArray[i].UpdatePlacement();

# DICOM reader
reader = vtkDICOMImageReader();
reader.SetDirectoryName('/data/VTK/TestVTK/bin/dcmdata/');
reader.Update();
imageDims = reader.GetOutput().GetDimensions();

# // Bounding box
outline = vtkOutlineFilter();
outline.SetInputConnection(reader.GetOutputPort());

outlineMapper = vtkPolyDataMapper();
outlineMapper.SetInputConnection(outline.GetOutputPort());

outlineActor = vtkActor();
outlineActor.SetMapper(outlineMapper);
outlineActor.GetProperty().SetColor(1,1,0);

# // Mapper and actors for volume
volumeMapper = vtkPolyDataMapper();
volumeMapper.SetInputConnection(reader.GetOutputPort());

volumeActor = vtkActor();
volumeActor.SetMapper(volumeMapper);

# // Renderers
renWin = vtkRenderWindow();
RendererArray = [None]*4;
for i in range(0,4):
    RendererArray[i] = vtkRenderer();
    renWin.AddRenderer(RendererArray[i]);
renWin.SetMultiSamples(0);

# // Render window interactor
iren = vtkRenderWindowInteractor();
renWin.SetInteractor(iren);

# // Picker
picker = vtkCellPicker();
picker.SetTolerance(0.005);

# // Properties
ipwProp = vtkProperty();

# // 3D plane widgets
planeWidgetArray = [None]*3;
for i in range(0,3):
    planeWidgetArray[i] = vtkImagePlaneWidget();
    planeWidgetArray[i].SetInteractor(iren);
    planeWidgetArray[i].SetPicker(picker);
    planeWidgetArray[i].RestrictPlaneToVolumeOn();
    color = [0, 0, 0 ];
    color[i] = 1;
    planeWidgetArray[i].GetPlaneProperty().SetColor(color);
    planeWidgetArray[i].SetTexturePlaneProperty(ipwProp);
    planeWidgetArray[i].TextureInterpolateOff();
    planeWidgetArray[i].SetResliceInterpolateToLinear();
    planeWidgetArray[i].SetInputConnection(reader.GetOutputPort());
    planeWidgetArray[i].SetPlaneOrientation(i);
    planeWidgetArray[i].SetSliceIndex(int(imageDims[i] / 2));
    planeWidgetArray[i].DisplayTextOn();
    planeWidgetArray[i].SetDefaultRenderer(RendererArray[3]);
    planeWidgetArray[i].SetWindowLevel(1358, -27, 0);
    planeWidgetArray[i].On();
    planeWidgetArray[i].InteractionOff();

planeWidgetArray[1].SetLookupTable(planeWidgetArray[0].GetLookupTable()); 
planeWidgetArray[2].SetLookupTable(planeWidgetArray[0].GetLookupTable());

# // ResliceCursor
resliceCursor = vtkResliceCursor();
center = reader.GetOutput().GetCenter();
resliceCursor.SetCenter(center[0], center[1], center[2]);
resliceCursor.SetThickMode(0);
resliceCursor.SetThickness(10, 10, 10);
resliceCursor.SetHole(0);
resliceCursor.SetImage(reader.GetOutput());

# // 2D Reslice cursor widgets
resliceCursorRepArray = [None]*3;
resliceCursorWidgetArray = [None]*3;
viewUp = [[0, 0, -1],[0, 0, -1],[0, 1, 0]];
for i in range(0,3):
    resliceCursorWidgetArray[i] = vtkResliceCursorWidget();
    resliceCursorRepArray[i] = vtkResliceCursorLineRepresentation();
    resliceCursorWidgetArray[i].SetInteractor(iren);
    resliceCursorWidgetArray[i].SetRepresentation(resliceCursorRepArray[i]);
    resliceCursorRepArray[i].GetResliceCursorActor().GetCursorAlgorithm().SetResliceCursor(resliceCursor);
    resliceCursorRepArray[i].GetResliceCursorActor().GetCursorAlgorithm().SetReslicePlaneNormal(i);
    
    for j in range(3):
        resliceCursorRepArray[i].GetResliceCursorActor().GetCenterlineProperty(j).SetRepresentationToWireframe()
        resliceCursorRepArray[i].GetResliceCursorActor().GetCenterlineProperty(j).RenderLinesAsTubesOn()
        resliceCursorRepArray[i].GetResliceCursorActor().GetCenterlineProperty(j).SetLineWidth(1)
                
    minVal = reader.GetOutput().GetScalarRange()[0];
    reslice = resliceCursorRepArray[i].GetReslice();
    reslice.SetInputConnection(reader.GetOutputPort());
    reslice.SetBackgroundColor(minVal, minVal, minVal, minVal);
    reslice.AutoCropOutputOn();
    reslice.Update();
    resliceCursorWidgetArray[i].SetDefaultRenderer(RendererArray[i]);
    resliceCursorWidgetArray[i].SetEnabled(1);
    RendererArray[i].GetActiveCamera().SetFocalPoint(0, 0, 0);
    camPos = [0, 0, 0];
    camPos[i] = 1;
    RendererArray[i].GetActiveCamera().SetPosition(camPos[0], camPos[1], camPos[2]);
    RendererArray[i].GetActiveCamera().ParallelProjectionOn();
    RendererArray[i].GetActiveCamera().SetViewUp(viewUp[i][0], viewUp[i][1], viewUp[i][2]);
    RendererArray[i].ResetCamera();
    resliceCursorWidgetArray[i].AddObserver('InteractionEvent',ResliceCursorCallback)
    range_color = reader.GetOutput().GetScalarRange();
    resliceCursorRepArray[i].SetWindowLevel(range_color[1] - range_color[0], (range_color[0] + range_color[1]) / 2.0, 0);
    planeWidgetArray[i].SetWindowLevel(range_color[1] - range_color[0], (range_color[0] + range_color[1]) / 2.0, 0);
    resliceCursorRepArray[i].SetLookupTable(resliceCursorRepArray[0].GetLookupTable());
    planeWidgetArray[i].GetColorMap().SetLookupTable(resliceCursorRepArray[0].GetLookupTable());
    
# // Background
RendererArray[0].SetBackground(0.3, 0.1, 0.1);
RendererArray[1].SetBackground(0.1, 0.3, 0.1);
RendererArray[2].SetBackground(0.1, 0.1, 0.3);
RendererArray[3].AddActor(volumeActor);
RendererArray[3].AddActor(outlineActor);
RendererArray[3].SetBackground(0.1, 0.1, 0.1);
renWin.SetSize(600, 600);

# // Render
RendererArray[0].SetViewport(0, 0, 0.5, 0.5);
RendererArray[1].SetViewport(0.5, 0, 1, 0.5);
RendererArray[2].SetViewport(0, 0.5, 0.5, 1);
RendererArray[3].SetViewport(0.5, 0.5, 1, 1);
renWin.Render();

# // Camera in 3D view
RendererArray[3].GetActiveCamera().Elevation(110);
RendererArray[3].GetActiveCamera().SetViewUp(0, 0, -1);
RendererArray[3].GetActiveCamera().Azimuth(45);
RendererArray[3].GetActiveCamera().Dolly(1.15);
RendererArray[3].ResetCameraClippingRange();

iren.Initialize()
iren.Start()

thank for you response,this is my test code,i have add defaultrenderer,but the promble is not resolved.

11

Thanks for sharing, your code looks correct. I confirm setting the default renderer won’t help as vtkResliceCursorPicker consider the intersection to be valid even when the provided point is outside the renderer.
I am not sure there is a way to fix this without modifying VTK. The right approach would be to early return in vtkResliceCursorPicker when the point is outside the viewport, or at least modify vtkResliceCursorPicker::IntersectPolyDataWithLine to return 0 when vtkCell::IntersectWithLine returns a parametric coord outside the [0 - tol, 1 + tol] range.
Maybe you can find a way to add observers on mouse events, and do the viewport check in your callback function to disable interactions on widgets outside the viewport.

Feel free to contact me directly if you would like Kitware to work on the fix in the context of a support contract.

12

Thanks,I have add observers on mouse events,and can get event from which view,but dont prohibition of running.some code as this:

   for i in range(3):
        self.resliceCursorWidget[i].AddObserver(vtk.vtkResliceCursorWidget.ResliceAxesChangedEvent,
                                                self.reslice_axes_changed_callback)

//this can save active view
def interaction_callback(self, obj, event):
click_pos = obj.GetEventPosition()
x, y = click_pos

    for renderer in self.renderWindow.GetRenderers():
        viewport = renderer.GetViewport()
        viewport_x = (x / self.renderWindow.GetSize()[0])
        viewport_y = (y / self.renderWindow.GetSize()[1])
        if viewport[0] <= viewport_x < viewport[2] and viewport[1] <= viewport_y < viewport[3]:
            self.active_widget = renderer
            #print(f"renderer: {renderer}")
            break
        else:
            print("no render view")

//this can get active line event,but i dont konw how to stop event
def reslice_axes_changed_callback(self, caller, event):
renderer = caller.GetCurrentRenderer()
#print(f"Reslice Axes Changed!: {renderer}")

    if self.active_widget == renderer:
        caller.InvokeEvent(event)
13

When you receive the widget events it’s too late as the representation has already been modified.
You need to catch the event before it gets processed by the widget. Observing mouse events and disabling interactions on every widget but the one inside
the renderer that is under your mouse pointer might work, but you will probably have to write a lot of code to handle this properly.
The best approach in the long term is probably to fix VTK, it should be as easy as adding the same IsInViewport check as done in vtkResliceCursorWidget::StartWindowLevel()

14

thanks，When can this issue be scheduled, or do I need to submit another topic?

15

You can open a new issue with steps to reproduce on the VTK gitlab repository, but unless you provide fundings for it, it’s unlikely that it will get fixed in the near future IMO.
You can also try to fix it yourself by giving a try to the approach I pointed in my previous message, and then open a merge request in VTK. I’ll be happy to review the change.

16

I will be try,thank you very much!

17

I have other question，about render window，when I move navigator lines，the view 3D volume also render meantime.this will take a long time to render volume，lead to slicer slower.how to avoid it,thanks

18

While VTK offers essential low-level features for implementing a medical image viewer, you need to spend very significant additional effort (dozens of developer years) to implement a full-featured medical imaging application that can be deployed to end users. Since there are VTK-based free, restriction-free, commercialization-friendly, open-source medical image computing platforms with proven track record of successful regulatory approval (e.g., 3D Slicer, MITK), it is hard to justify redoing all this work from scratch. So, if your goal is to have a full-featured medical image viewer then I would recommend to stop thinking about reimplementing basic features and build your application by customizing/extending an existing platform.

If your goal is learning programming in the medical imaging domain then it may be useful to work on small toy projects. However, VTK widgets may not really worth the time. Mostly because they are not widely used, they are complex yet have very limited feature set, their implementation is often incomplete (basic features are missing or not working robustly), and they are not visually appealing. Instead, you can focus your efforts on getting experience with various processing and visualization algorithms.

If you want to “just implement something simple”, then nowadays it is not an option to implement a desktop application for that, because users expect to do simple things in a web browser, maybe in a mobile app. If this is the case then I would recommend to check out trame, VTK webassembly, VTK.js examples to see if the simple functionalities that you need are readily available.

1 Like
19

Thanks for your patient response. I’m currently working on a small demo to verify and learn some features of VTK, and I’m unsure whether some issues I’m encountering are due to usage errors or defects within VTK itself. I’ve been posting my questions to seek help from experts.

20

Just to answer this question, you must always render the whole scene, it’s impossible to skip specific objects. The best you can achieve is to lower the volume rendering quality by increasing the sampling distance during interactions. The default AutoAdjustSamplesDistance and DesiredUpdateRate/StillUpdateRate parameters should handle it already.
For further requests, please open new topics when the question changes.