Hi,
I did some performance profilling for the method vtkImageViewer2.SetSlice(int), and found that method consumed large amounts of CPU usage, especailly when continually setting different slices. Is this a known issue? Or any solutions to avoid the performance bottleneck? Thanks!
This is expected. Image reslicing is a very costly operation and it is often a performance bottleneck.
If you reslice along image axes, make sure that you use exactly the same axis directions (there is faster computation path if image axes are exactly parallel with reslice axes). You may also try to adjust number of threads (using fewer threads may make reslicing faster in some cases).
If you are looking for more dramatic speed improvement (100x faster) then you need to store the volume on GPU and reslice it there. I don’t know if there is a dedicated mapper for this, but if not then you can use GPU volume rendering mapper (with a region of interest that is limited to a thin slice).
Have you ever seen this in action? As far as I understand, volume rendering a single slice will not work.
This approach is used in the new OHIF web viewer and it is embarrassingly fast compared to our CPU-based reslicing in Slicer, especially when thick slab reslicing is used.
Currently, you need at least two slices in the volume for volume rendering to work, but this should be possible to fix in the volume renderer and you can easily work it around for the special case of single-slice volumes (duplicate the slice and use volume mapper; or use a simple 2D image mapper).
Very interesting. OHIF uses VTK’s volume rendering? And if I may be so bold, why hasn’t the Slicer community developed a GPU-based slice mapper for VTK?
OHIF latest development version now uses vtk.js for rendering.
We may eventually get to it but it has not been as high priority for us as other things, such as developing 3D widgets framework (point set, line, curve, angle, plane, etc. widgets), integrating new rendering features (such as multi-volume rendering), and lately just getting VTK9 to work at all (there has been lots of regressions/differences in behavior compared to 8.2). It also requires a few more things, such as sharing of volume data between all the renderers and developing a GPU image blend filter (to be able to composite foreground, background, and labelmap layers).
I definitely agree that the VTK widget infrastructure needs an overhaul (but I don’t want to get too far off topic).
For fast 3D slicing, I think that the easiest approach would be to implement support for 3D data with vtkTexture. All that is needed is 3D texture mapping onto a rectangle, assuming that vtkTexture will support 16-bit int or float data.
Hi @dgobbi -
You can see it in action at the Imaging Data Commons (IDC) portal. Press the 2D MPR button to see it in action it should look like the images below.
In OHIF we don’t render a thin slice so much as we set the clipping plane to be the MPR plane and set the scalar opacity to 1 everywhere. So it looks like a reslice but it’s really a volume rendering, which is why slab MIP is a natural extension. We have some new work coming with PET/CT compositing and more tools.
Having written the initial implementations of both the Slicer and OHIF MPR modes, I can say there are definite tradeoffs between the two approaches.
First off, the GPU is definitely a more limited resource than the CPU. For example, the IDC contains one lower extremity runoff series of 3000 slices that cannot be allocated in the GPU, but of course that would be fine in most CPUs.
So for Slicer we don’t want to support a wide range of use cases and target platforms and the CPU makes a lot more sense.
In addition of course, as Andras mentioned, we support a lot more functionality in Slicer. One thing we really leverage from C++ VTK is your transform code to allow nonlinear reslicing, which isn’t supported currently in vtk.js.
Rather than extend vtkTexture in C++ VTK, probably better to work on the multivolume volume renderer and support more options in the shader code (e.g. transforms).
Yes, we need a lot of features from volume rendering (color and opacity mapping, thick slab reslicing, cropping, etc.), so even if it is not the simplest solution to use the volume renderer for slice rendering in the long term it could pay off. Advanced features, such as real-time non-linear transforms, handling of very large data sets (with dynamic retrieval of data from the best suited multi-resolution pyramid), etc. would also all need to be developed only once.
Thanks for the info, Steve. In fact I’ve looked through the vtk.js vtkImageReslice code and hope to go deeper into vtk.js in the future (time permitting).
Yes, the 3D Slicer implementation of this is all in C++ and is open source and reusable in your context. Of course the code is a bit complex since it is pretty general purpose.
@pieper I am willing to take a stab at it. How do I get started. I have the Slicer code downloaded. Any help hints that would save me time? Do you have code isolated that you could make available? How modular it it?
Hi @dporter - Since we have a parallel thread on Slicer discourse let’s take up the Slicer-specific details there. But to answer the toplevel question, Slicer uses the term “Logic” to refer to the vtk classes that implement all the reslicing. The “GUI” layer is where the Qt implementation is, so that’s all mutable.
Is there a way to set the initial thickness or change the thickness of the VTKResliceImageViewer prior to of during a user-interactive thick slab mode cross reference session?
@sankhesh @dgobbi My apologies. I’ll be more clear. For example, how do I set the slice thickness to 4.0mm or 2.0mm, etc., without using the interface or cursor handles? My physicians are requiring a user preferred default thickness and to be able to enter the desired thickness via text box or slider. Thanks.
@dgobbi, I am using your vtkDICOMReader and for the life of me, I can’t orient the 3 views the way my radiologists want them.
A S S
R L A P R L
P I I
Axial Sagittal Coronal
Is this possible? My apologies in advance for my ignorance. If these are more complex issues than I am assuming, please let me know so that I know it will require deeper study. Thanks!
Hi Doug, the orientations of vtkResliceImageViewer are hard-coded in its UpdateOrientation() method. If the orientations that it provides are not the ones that you require, then your best bet is to make your own subclass of vtkResliceImageViewer and override the UpdateOrientation() method to get the orientations you need.