This sounds awesome. Really looking forward to more realistic-looking lighting for volume rendering.
I cannot interpret the images posted at the top. The volume rendering settings do not seem to be set to for visualizing either bones or any soft tissues.
Example rendering results that we would like to get
CT rendering - from https://discourse.vtk.org/t/bvtknodes-photorealistic-rendering-of-vtk-data-in-blender/3268/14:
CT rendering - from https://discourse.vtk.org/t/bvtknodes-photorealistic-rendering-of-vtk-data-in-blender/3268/16:
3D obstetric ultrasound rendering - from https://www.philips.co.uk/healthcare/resources/feature-detail/ultrasound-truevue-imaging:
3D cardiac ultrasound rendering - from https://www.philips.com/a-w/about/news/archive/blogs/innovation-matters/20191113-five-innovations-that-are-shaping-the-future-of-image-guided-therapy.html:
The 3D cardiac CT image that is rendered above is available here. I can also provide 4D cardiac ultrasound volumes that should be possible to render similarly as the 3D cardiac ultrasound rendering example above.
Potential ideas for improvements
The ideas behind the new illumination methods were not described in this post, and the renderings posted so far are not very informative yet, so I cannot comment on the proposed new methods. instead, I have two suggestions for hugely improving volume rendering appearance, without increasing rendering time, and requiring only small development effort.
Since surfaces and their normal can be estimated directly from 3D volumes, it may be feasible to port these two VTK rendering features that are currently available only for surface rendering:
- Screen-space ambient occlusion (SSAO): it would serve as a very simple, inexpensive approximation of complicated global illumination methods (https://discourse.slicer.org/t/new-feature-surface-rendering-with-ambient-shadows-for-improved-depth-perception/16903)
- Image-based lighting: for example a simple MatCaps shader can be applied directly to volume rendering (see https://discourse.slicer.org/t/how-to-perform-3d-cinematic-rendering/7313/7, @Chris_Rorden even shared the source code of the shader):
