Hey everyone! I recently had a CT Coronary scan done, and I’ve been exploring ways to visualize the data using the VTK ecosystem. I have the DICOM series from the scan and have successfully generated a 3D volume for visualization (ended up using pydicom for this), which I think could serve as a great example to show off the power of VTK’s volume rendering capabilities.
I’m not well versed on medical imaging datasets and feel like there is so much more I can do here in terms of processing/analysis to create some stunning visuals and would greatly appreciate any pointers the community may have.
If there’s interest, I am happy to provide my dataset for community projects and have presently uploaded the raw volume as a vtkImageData file here.
Overall, I’m hoping to create some visually stunning examples that demonstrate what’s possible within the VTK ecosystem. I would greatly appreciate any thoughts/pointers on tools or methods that could improve the segmentation and visualization process or any ideas on how to make this an even better showcase for VTK (and PyVista obviously because it’s me after all )
I’m struggling to really make great use of the scalar values from the CT scan and I feel like they may need a few transformations to really highlight certain features.
I’m also inspired by some other examples like the following and want to create something that looks somewhat realistic - it’d be really awesome if I could come up with a scalar mapping that maps tissues and bones, etc, to realistic colors and create something spooky just in time for Haloween.
Segmentation
In addition to volume rendering, I’ve experimented with segmenting the scan using the TotalSegmentator software. However, the results were a bit blocky/voxelized, and I’m interested in going further to extract more detailed structures, particularly some of the vasculature that is faintly visible in the data. I’m going to keep running with this as I think I can generate isosurfaces and then use taubian smoothing to create a nice effect.
One thing that would be really cool would be to segment the vasculature and create visuals of these networks like I’ve seen with VesselVio (though I think this software is geared towards brain scans)
I’m quite shocked that you don’t even mention 3D Slicer, which probably currently provides the best VTK-based medical image visualization methods.
Classic VTK volume rendering for medical images includes enabling shading and using carefully designed scalar and color opacity transfer functions, maybe some gradient opacity mapping. These already provide much better visualizations that you show above. You can get these by loading drag-and-dropping your CT into 3D Slicer, loading the images, drag-and-dropping into the 3D view, and play with settings in Volume Rendering module:
If you are interested in a particular region then you may consider using the Segment Editor module to blank out parts of the image that occlude your region of interest (e.g., Paint or Scissors effect, followed by with Mask Volume):
However, since full-body CT segmentation is nowadays is very easily available and not too long ago we have added screen-space ambient occlusion for volume rendering, you can now do much better than this. You can use TotalSegmentator or MONAIAuto3DSeg to show muscles, organs, and vessels with great details (that previously people did not even realize it was feasible to show in CT images).
Resolution of these automatic AI segmentations is lower than the original CT (1mm instead of 0.2-0.5mm), but it does not matter much, as the volume colorization method available in 3D Slicer’s Colorize volume module (in Sandbox extension) can keep all the details from the original CT and just use the segmentation for soft masking and coloring:
Finally, by enabling screen-space ambient occlusion for volume rendering, you can reveal additional texture and subsurface details (in Lights module, provided by Sandbox extension):
Silly of me to have not tried 3D Slicer!! Thanks for these suggestions and outline on using 3D slicer!! I managed to at least get my DICOM data imported and will definitely see what I can do here. Thanks!
)
