I use VTK to display medical images like CT and MR, which in the medical imaging domain are high-resolution imaging modalities. My problem is that when I render these images linear artifacts along pixel boundaries begin to appear and are very stark when zoomed to the pixel level.
They seem to arise from a limitation of the vtkImageActor to only permit linear interpolation in their texture maps. I found this old VTK thread, that seemed related.
In it the respondent indicates that this limitation is imposed by OpenGL as not all version support cubic interpolation. While this thread is 15 years old I don’t see any indication from the VTK documentation that the situation has changed.
Is this issue a problem for other users? Are there any intentions for VTK developers to extend the OpenGL texture maps to include cubic (or higher order) interpolation?
Thanks for your reply. I’ve attached a screen shot of a high res CT image from the application I am working on as well as 3D slicer. The images probably don’t show the same regions but they should suffice as an example of textures.
A screen shot from the application I am working with:
The linear boundary artifacts manifest in both, although it seems there is some different sources of smoothing between the two applications.
Below is the output of another application that does not use VTK where cubic interpolation is employed on the texture maps that softens these boundaries.
When the whole image is the visible the effects are very subtle but noticeable by some.
I think we just use simple linear interpolation in Slicer because this has not been reported as a problem over the last 20 years and thousands of medical image computing projects. We can easily make cubic interpolation option available from the GUI or maybe switch to cubic interpolation automatically when a certain zoom factor is reached.
In 3D Slicer, we perform our image reslicing and layer blending pipeline on CPU. This has many advantages: no need to transfer the volume to GPU memory (there is no risk that we cannot visualize a volume because it does not fit into GPU memory), we can display non-linearly transformed volumes without warping the entire 3D volume (just by passing the transformation pipeline to the reslice filter), use standard VTK filters for blending and other processing, etc.
Of course running this pipeline on the GPU it is considerably slower than the GPU, but since on a typical computer and typical images the speed is acceptable (about 50fps on an i7 laptop for a 512^3 volume), we are not in a rush to move everything to the GPU.
That’s very interesting, it does seem to give the desired effect.
We use the vtkImageReslice class as part of out visualization pipeline to stack through 3D image volumes as well as to produce mutli-planar reformatted images. The interpolation method of the vtkImageReslice object appears to have no affect on the linear interpolation that appears in the displayed image.
Only when I toggle on/off the vtkImageActor interpolation do I notice a change from linear interpolation to (what appears to be) nearest neighbour. When the image actor’s interpolation is off the rendered pixels have a flat (uniform) intensity.
Our pipeline was configured such that we don’t use the vrkImageReslice to resize the image to zoom, instead we use the vtkCamera. Can you comment on how zoom is performed in 3D Slicer? Are the images truncated and up-sampled (or resized) using the vtkImageReslice filter?
Currently, the most reliable method for displaying images with cubic or higher-order interpolation is vtkImageResliceMapper. This mapper internally uses vtkImageReslice for any zoom operations that are performed, in addition to the slicing. Of course, this can cause zooming to be slower than for vtkImageActor.
The vtkImageResliceMapper::SetInterpolator() method allows you to specify vtkImageInterpolator object that gives more control over the interpolation. There is also a vtkImageSincInterpolator and vtkImageBSplineInterpolator (though the latter requires that the image be pre-filtered with vtkImageBSplineCoefficients).