Issues with Lighting and Clipping in Multi-Renderer VTK.js Setup

Web
1

Hello,
Recently I tried using multiple renderers within the same renderWindow: one for displaying a volume in the background and another for displaying polydata elements in the foreground. However, I encountered some issues:

  • The lighting doesn’t seem to follow the camera correctly in the scene.
  • The camera’s clipping range appears to only take the foreground elements into account, ignoring the volume in the background.

light_fixed
light_fixed541×494 144 KB

clipping_range.PNG
clipping_range.PNG718×698 35.1 KB

Here is how I create the renderers:

// Create render window
const genericRenderWindow = vtkGenericRenderWindow.newInstance();
genericRenderWindow.setContainer(containerRef.current);
genericRenderWindow.resize();

const renderWindow = genericRenderWindow.getRenderWindow();

// Use default renderer as background renderer (layer 0)
const backgroundRenderer = genericRenderWindow.getRenderer();
backgroundRenderer.setLayer(0);
backgroundRenderer.setBackground(0.341, 0.325, 0.306);
backgroundRenderer.setBackingStore(true);

// Create additional renderer for foreground layer
const foregroundRenderer = vtkRenderer.newInstance();
const interactor = renderWindow.getInteractor();

// Add foreground renderer to render window
renderWindow.addRenderer(foregroundRenderer);
interactor.getInteractorStyle().setFocusedRenderer(backgroundRenderer);

foregroundRenderer.setLayer(1);
renderWindow.setNumberOfLayers(2);

foregroundRenderer.setPreserveDepthBuffer(true);
foregroundRenderer.setPreserveColorBuffer(true);

// Share the same camera between renderers
foregroundRenderer.setActiveCamera(backgroundRenderer.getActiveCamera());

rendererRef.current = {
  background: backgroundRenderer,
  foreground: foregroundRenderer,
};

To add the volume to the background renderer:

// Create volume and mapper
const volume = vtkVolume.newInstance();
const mapper = vtkVolumeMapper.newInstance();

mapper.setInputData(maskedVtkImage);
volume.setMapper(mapper);

const volumeProperty = volume.getProperty();

// Apply transfer functions
const transferFunctions = createTransferFunctions3D();
volumeProperty.setScalarOpacity(0, transferFunctions.opacityFunction);
volumeProperty.setRGBTransferFunction(0, transferFunctions.colorFunction);

rendererRef.current.background.addVolume(volume);

To add an actor to the foreground renderer:

const sphereSource = vtkSphereSource.newInstance({
  center: [0, 0, 0],
  radius: 2.2,
});

const sphereMapper = vtkMapper.newInstance();
sphereMapper.setInputConnection(sphereSource.getOutputPort());

const sphereActor = vtkActor.newInstance();
sphereActor.setMapper(sphereMapper);
sphereActor.getProperty().setColor(0, 1, 0); // Green

if (position) {
  sphereActor.setPosition(position.x, position.y, position.z);
}

rendererCurrentRef.addActor(sphereActor);

Interestingly, when I add the following code using the orientation widget, it seems to fix the issues:

import {
  createInteractiveOrientationMarkerWidget,
  alignCameraOnViewWidgetOrientationChange,
} from '@kitware/vtk.js/Widgets/Widgets3D/InteractiveOrientationWidget/helpers';

const { interactiveOrientationWidget, orientationMarkerWidget } =
  createInteractiveOrientationMarkerWidget(
    globalWidgetManager,
    renderWindowInteractor,
    renderer
  );
2

It’s not clear to me what the problem is.
The volume rendering does not write in the depth buffer, that explains why the meshes render “on top” of the volume rendering.
I can’t see any problem with the clipping ranges. Same for the light.

You might want to try the beta branch of VTK.js, there may have some volume rendering fixes in there.

Hth,
Julien.

4

Hi Julien,

Thanks for your reply. I haven’t looked into the beta branch yet.

To clarify the issue: when I render only the volume using a single renderer, the lighting correctly follows the volume, and I never get any clipping when moving the camera (highlighted in blue in the second image). However, as soon as I add a second renderer, it seems like the light remains fixed above, and the clipping range adjusts based on the objects in my foreground. Here’s an image showing the issue resolved when I use the orientation widget.

image
image561×654 66.1 KB

Guillaume

5

For the clipping, it might be because both renderers share the same camera. But both renderer modify the camera based on their own bounds. The foreground renderer may modifiy the camera bounds, and this has an impact when rendering the background renderer. A solution could be to have 2 different cameras and “copy” the background camera (position+focal point) onto the foreground camera.
Maybe that will also fix the lighting as well.

6

Hello,
I checked the code in the file Widgets/Widgets3D/InteractiveOrientationWidget/helpers.js.

It seems that the following line could resolve the issue I’m experiencing when placed after setting the interactorStyle:

interactor.getInteractorStyle().setFocusedRenderer(renderer);

useEffect(() => {
    if (!ready.renderer || !ready.renderWindow) return;

    const interactor = renderWindow.getInteractor();
    interactor.setDesiredUpdateRate(15.0);
    const interactorStyle = vtkInteractorStyleManipulator.newInstance();

    updateRotationCenter(interactorStyle, renderer);

    interactorStyleRef.current = interactorStyle;

    interactor.setInteractorStyle(interactorStyle);
    interactor.getInteractorStyle().setFocusedRenderer(renderer); // background renderer

    setReady(prev => ({ ...prev, interactor: true }));
}, [viewmode, ready.renderer, ready.renderWindow, setReady]);

However, this only takes effect after I move the actor.