vtk.js: Image renders black, but Cube renders fine – vtkImageData + vtkImageSlice issue

Web
1

I’m using vtk.js inside a React component to render a 2D image slice using vtkImageData and vtkImageSlice. While 3D geometry primitives like vtkCubeSource render correctly (e.g., a cube appears as expected), my 2D image shows up completely black — even though the scalar data appears valid and I’ve configured color and opacity transfer functions.

I’ve created a React hook that initializes the VTK rendering pipeline and draws the image to the renderer. The vtkImageDataobject represents a 2D image. Despite it holds expected structure, the image does not appear, or renders as black. Meanwhile, the cube geometry appears without issues in the same scene, suggesting that the renderer and camera setup are generally functional. This is the code for the react hook:


const sceneRef = useRef(null);
const setupView = (viewContainer, vtk_imageData) => {  
  // Scene setup to reuse resources
  if (!sceneRef.current) {
    sceneRef.current = {}
    const fullScreenRenderer = vtkFullScreenRenderWindow.newInstance({
      rootContainer: viewContainer,
      background: [0.1, 0.1, 0.1],
    });
    //Set up renderer
    const renderer = fullScreenRenderer.getRenderer();
    const renderWindow = fullScreenRenderer.getRenderWindow();
    const camera = renderer.getActiveCamera();

    // setup 2D view
    camera.setParallelProjection(true);
    const iStyle = vtkInteractorStyleImage.newInstance();
    iStyle.setInteractionMode('IMAGE_SLICING');
    renderWindow.getInteractor().setInteractorStyle(iStyle);
    sceneRef.current = {
      fullScreenRenderer,
      renderer,
      renderWindow,
      camera,
      iStyle,
    };
  }

  const scene = sceneRef.current;
  console.log("scene:", scene);
  console.log("vtk_imageData:", vtk_imageData);

  scene.image = {
    imageMapper: vtkImageMapper.newInstance(),
    actor: vtkImageSlice.newInstance(),
  };

  // background image pipeline
  scene.image.actor.setMapper(scene.image.imageMapper);

  // Trying to configure color transfer function and piecewiseFunction ... but does not work without it as well (black to white)
  const range = vtk_imageData.getPointData().getScalars().getRange();
  const imageProp = scene.image.actor.getProperty();
  const ctf = vtkColorTransferFunction.newInstance();
  ctf.addRGBPoint(0, 0.0, 0.0, 0.0);
  ctf.addRGBPoint(255, 1.0, 1.0, 1.0);
  const pf = vtkPiecewiseFunction.newInstance();
  pf.addPoint(range[0], 1.0);
  pf.addPoint(range[1], 1.0);
  
  imageProp.setRGBTransferFunction(0, ctf);
  imageProp.setScalarOpacity(0, pf);
  imageProp.setUseLookupTableScalarRange(true);
  // End playing with transfer functions

  scene.image.imageMapper.setInputData(vtk_imageData);

  // add actors to renderers ... both ways do nothing
  //scene.renderer.addViewProp(scene.image.actor);
  scene.renderer.addActor(scene.image.actor);

  // default slice orientation/mode and camera view
  const sliceMode = vtkImageMapper.SlicingMode.K;
  scene.image.imageMapper.setSlicingMode(sliceMode);
  scene.image.imageMapper.setSlice(0);
  scene.image.imageMapper.update();

  // set 2D camera position
  setCamera(sliceMode, scene.renderer, vtk_imageData);


  //Last try
  const ijk = [0, 0, 0];
  const position = [0, 0, 0];
  ijk[sliceMode] = scene.image.imageMapper.getSlice();
  vtk_imageData.indexToWorld(ijk, position);
  //Still without effect


  // Create cube geometry -> Interestingly this works
  const cubeSource = vtkCubeSource.newInstance({
    xLength: 100,
    yLength: 200,
    zLength: 10,
    center: [0, 0, 0],
  });
  const cubeMapper = vtkMapper.newInstance();
  cubeMapper.setInputConnection(cubeSource.getOutputPort());
  const cubeActor = vtkActor.newInstance();
  cubeActor.setMapper(cubeMapper);
  
  
  scene.renderer.addActor(cubeActor);
  // Cube shows up but not 2D image
  scene.renderWindow.render();
};
    const initExample = async () => {
      try {
        const reader = vtkHttpDataSetReader.newInstance({ fetchGzip: true });
        const url = 'https://raw.githubusercontent.com/Kitware/vtk-js/master/Data/volume/LIDC2.vti/'
        await reader.setUrl(url, { loadData: true });
        const vtkImage = reader.getOutputData();

        // Now call your existing setup function with the vtkImage
        setupView(viewRef.current, vtkImage);
      } catch (err) {
        console.error('Failed to load VTK dataset:', err);
      }
    };

    //Initialize and draw image
    //init();
    initExample();


    // Cleanup function to avoid setting state after unmount
    return () => {
    };

  }, [viewRef]);

  return (
    <div
      ref={viewRef}
      className="flex-1 h-full w-full"
      style={{
        border: debug ? "5px solid red" : "none",
      }}
    />
  );
}

Complete source of the defective component is here:

Note: My ultimate goal is to transition the Viv project from using deck.gl and luma.gl to vtk.js, in order to gain more granular control over rendering and potentially unlock improved 3D visualization capabilities. To the best of my knowledge, vtk.js does not natively support formats like Zarr that enable efficient HTTP range requests. Therefore, I’m attempting to integrate both, using Zarr-based loading for large datasets, while leveraging vtk.js for the rendering pipeline.

2

You need to import the Image profile in addition to the Geometry profile