Hello again,
Thank you @sankhesh for the detailed explanation and feedback. I love it !
I completely understand the issue with depth values in volume rendering. Maybe I’ll start with my project goal, which was described here also (Volume Rendering Stereo Depth). However, it is now deprecated, so I’ll try to describe it better again.
Project Goal:
In brief, I need to share a VTK render texture into Unity and mix it with Unity scene content, which is why I need both color and depth images. Importantly, the VTK scene contains just a volume.
Current Implementation:
In my stereo rendering setup, each renderer is equipped with its own dedicated volume mapper, which operates in render-to-image mode. I override Unity’s Z-buffer with the depth map from VTK, and to be honest (with slicer transfer functions) that is completely sufficient for an experience as far as it is more like in the case on the left side of your screenshot Attaching SS of the result.
Problem:
I tried to integrate the SSAO feature (which is awesome, by the way!) but it is not available with render-to-image mode.
Possible Solutions:
Either write depth information to the Z-buffer in render-to-window mode or enable SSAO in render-to-image mode. I have a feeling that the first solution should be easier to achieve.
I’d be grateful for a tip on how to mix SSAO with getting depth information together.
PS.
Is SSAO using the same depth values that render-to-image mode produces? It’s very interesting that even though selecting depth values from the ray marching algorithm is not straightforward, SSAO does such great work here. Also, it is a bit odd to me that you need depth information in SSAO but you don’t have access to its map, but maybe I’m missing something here.