[VTK.js/React] Unable to smooth ImageData using ImageMarchingCubes and WindowedSyncPolyDataFilter

Web
1

I’m running into some issues trying to smooth the ImageData coming from a Niftii file.

When I render the ImageData without the ImageMarchingCubes and WindowedSyncPolyDataFilter, everything works as expected but the ImageData is not smooth (stair-stepping occurs). That’s where MarchingCubes and WindowedSyncPolyDataFilter should come in. Unfortunately, applying ImageMarchingCubes and WindowedSyncPolyDataFilter throws the following error:
Uncaught (in promise) TypeError: WebGL2RenderingContext.bindTexture: Argument 2 is not an object.

I’ve been stuck on this issue for the past few days now and I can’t figure out what’s happening or how to fix this. I’ve tried replacing RenderWindows with OpenGLRenderWindows, I’ve tried adding an OpenGLRenderWindow as view, I’ve tried adding PolyDataNormals, but the error keeps appearing, either immediately on render (after applying ImageMarchingCubes and WindowedSyncPolyDataNormals) or when I interact with the window (with no visualisation happening). Can anyone help me out?

Thanks in advance!

My code (it’s possible there are some syntax issues or missing imports since I’ve edited my actual code to be more readable for this forum):

import { useEffect } from 'react';

// MUI imports
import {
    Container,
    Grid,
} from '@mui/material';

// VTK imports.
import '@kitware/vtk.js/Rendering/Profiles/Geometry';
import '@kitware/vtk.js/Rendering/Profiles/Volume';
import vtkVolume from '@kitware/vtk.js/Rendering/Core/Volume';
import vtkVolumeMapper from '@kitware/vtk.js/Rendering/Core/VolumeMapper';
import vtkGenericRenderWindow from '@kitware/vtk.js/Rendering/Misc/GenericRenderWindow';
import vtkColorTransferFunction from '@kitware/vtk.js/Rendering/Core/ColorTransferFunction';
import vtkPiecewiseFunction from '@kitware/vtk.js/Common/DataModel/PiecewiseFunction';
import vtkColorMaps from '@kitware/vtk.js/Rendering/Core/ColorTransferFunction/ColorMaps';
import vtkImageMarchingCubes from '@kitware/vtk.js/Filters/General/ImageMarchingCubes';
import vtkWindowedSincPolyDataFilter from '@kitware/vtk.js/Filters/General/WindowedSincPolyDataFilter';
import vtkITKHelper from '@kitware/vtk.js/Common/DataModel/ITKHelper';
import vtkLiteHttpDataAccessHelper from '@kitware/vtk.js/IO/Core/DataAccessHelper/LiteHttpDataAccessHelper';

// ITK imports.
import { readImageArrayBuffer } from 'itk-wasm';

const App = () => {

    useEffect(() => {
        createWindow();
    }, []);

    const readFile = async (file) => {
        const arrayBuffer = await vtkLiteHttpDataAccessHelper.fetchBinary(file.path);

        const { image: itkImage, webWorker } = await readImageArrayBuffer(
            null,
            arrayBuffer,
            file.name,
        );
        webWorker.terminate();

        return vtkITKHelper.convertItkToVtkImage(itkImage);
    };

    const update = async (actor, renderer, renderWindow, imageData) => {
        // Set up color lookup table and opacity piecewise function.
        const lookupTable = vtkColorTransferFunction.newInstance();
        const piecewiseFun = vtkPiecewiseFunction.newInstance();

        // Set up color transfer function.
        lookupTable.applyColorMap(vtkColorMaps.getPresetByName('bone_Matlab'));

        // Initial mapping range.
        const range = imageData.getPointData().getScalars().getRange();
        lookupTable.setMappingRange(...range);
        lookupTable.updateRange();

        // Set up simple linear opacity function.
        // This assumes a data range of 0 -> 256.
        for (let i = 0; i <= 8; i++) {
            piecewiseFun.addPoint(i * 32, i / 8);
        }

        // Set the actor properties.
        actor.getProperty().setRGBTransferFunction(0, lookupTable);
        actor.getProperty().setScalarOpacity(0, piecewiseFun);
        actor.getProperty().setInterpolationTypeToLinear();

        // Add volume actor to scene.
        renderer.addVolume(actor);

        // Update lookup table mapping range based on input dataset.
        lookupTable.setMappingRange(...range);
        lookupTable.updateRange();

        // Reset camera and render scene.
        renderer.resetCamera();
        renderer.updateLightsGeometryToFollowCamera();
        renderWindow.render();
    };

    const createWindow = async () => {
        const container = document.querySelector('.container');
        const genericRenderWindow = vtkGenericRenderWindow.newInstance({
            background: [0, 0, 0],
        });
        genericRenderWindow.setContainer(container);
        genericRenderWindow.resize();

        const renderer = genericRenderWindow.getRenderer();
        const renderWindow = genericRenderWindow.getRenderWindow();

        const actor = vtkVolume.newInstance();
        const mapper = vtkVolumeMapper.newInstance();
        actor.setMapper(mapper);

        const file = {
            name: 'test.nii',
            path: 'data/test.nii',
        };

        const imageData = await readFile(file);

        // This is where the issues happen.
        const mCubes = vtkImageMarchingCubes.newInstance({
            contourValue: 0.5,
            computeNormals: true,
            mergePoints: true,
        });
        mCubes.setInputData(imageData);

        const smoothFilter = vtkWindowedSincPolyDataFilter.newInstance();
        smoothFilter.setInputData(mCubes.getOutputData());

        // mapper.setInputData(imageData); --> This works.
        mapper.setInputData(smoothFilter.getOutputData()); // --> This throws an error.

        update(actor, renderer, renderWindow, imageData);
    };

    return (
        <Container>
            <Grid container>
                <Grid item>
                    <div
                        className="container"
                        style={{ width: '400px', height: '400px' }}
                    ></div>
                </Grid>
            </Grid>
        </Container>
    );
};

export default App;
2

Use setInputConnection instead of setInputData whereever you can. Otherwise you need to call Update before calling getOutputData()

3

Hi Julien

Thank you for your input!

Just to be clear: with “call Update before calling getOutputData()”, you’re referring to calling smoothFilter.update() before calling smoothFilter.getOutputData(), as is the case with the Python version? Like this?

const mCubes = vtkImageMarchingCubes.newInstance({
    contourValue: 0.5,
    computeNormals: true,
    mergePoints: true,
});
mCubes.setInputData(imageData);

const smoothFilter = vtkWindowedSincPolyDataFilter.newInstance();
mCubes.update();
smoothFilter.setInputData(mCubes.getOutputData());

smoothFilter.update();
mapper.setInputData(smoothFilter.getOutputData());

Doing that still throws the error I’ve mentioned in my original post.

When I call my custom update method, a No input! error throws.

I also tried changing setInputData() and getOutputData() to setInputConnection() and setOutputPort() where possible. The code I used:

const mCubes = vtkImageMarchingCubes.newInstance({
    contourValue: 0.5,
    computeNormals: true,
    mergePoints: true,
});
mCubes.setInputData(imageData);

const smoothFilter = vtkWindowedSincPolyDataFilter.newInstance();
smoothFilter.setInputConnection(mCubes.getOutputPort());

mapper.setInputConnection(smoothFilter.getOutputPort());

Doing that still throws the error I’ve mentioned in my original post. I’ve also tried some other things, but still nothing.

If you have any other suggestions, I’d be happy to try them out. Is it possible this is a bug?

Thanks again.

4

Update: the issue turned out to be the vtkVolume and vtkVolumeMapper. Those should be vtkActor and vtkMapper, since I’m not trying to render a volume but a smooth surface.

In my case, using the code in my original post, that still threw an error. This error was related to the lookuptable and piecewisefunction I used to color the volume (that I no longer had after applying smoothing). Removing the code to apply colors to a volume rendered a smooth surface.