Is it possible to combine two inputs into one while rendering in MultiSliceImageMapper?

[Yuvi-416 (DMC-416-/)]

Hello everyone,

I am trying to render two images into one using the MultiSliceImageMapper algorithm. I have one image with two channels and want to render them in MultiSliceImageMapper. I can perform MultiSliceImageMapper for one however when I pass two images at the same time then it creates two canvases, however, Slice I, K, and Y control seem fine as it controls both images (the only problem is that, it didn’t render in one canvas and I want to fix that). As I am not getting any errors and I also don’t know whether it’s possible in this MultiSliceImageMapper case. I can do it in Volume Rendering by providing render.addActor().

In MultiSliceImageMapper, I do the same render.addActor() for adding ImageSliceK, J, and I. Now, I don’t know how can I add another image to it.

For example:
As in the below image, u can see that the two images is not merger/combined however it rendered in two canvas. Can I merge it and render it into one canvas? I didn’t find this problem in vtkVolume.

[Forrest]

You must be creating a new canvas somewhere, since the MultiSliceImageMapper example just uses 3 vtkImageMapper instances. I’d check to see if you are creating a whole new rendering pipeline by accident.

[Yuvi-416 (DMC-416-/)]

Hello Forest,
Sorry for the late reply.
I also checked whether I am creating another canvas somewhere but I don’t see any wrong in the code.

However, here’s the code:

function tri_planar(tri_imageData){

var container = document.getElementById('viewContainer');
const openglRenderWindow = vtk.Rendering.OpenGL.vtkRenderWindow.newInstance();
const renderer = vtk.Rendering.Core.vtkRenderer.newInstance();
const renderWindow = vtk.Rendering.Core.vtkRenderWindow.newInstance();
const vtkInteractorStyleTrackballCamera = vtk.Interaction.Style.vtkInteractorStyleTrackballCamera;
const vtkRenderWindowInteractor = vtk.Rendering.Core.vtkRenderWindowInteractor;
const vtkAnnotatedCubeActor = vtk.Rendering.Core.vtkAnnotatedCubeActor;
const vtkOrientationMarkerWidget = vtk.Interaction.Widgets.vtkOrientationMarkerWidget;
const interactor = vtk.Rendering.Core.vtkRenderWindowInteractor.newInstance();

const vtkInteractorStyleManipulator = vtk.Interaction.Style.vtkInteractorStyleManipulator;

const vtkImageSlice = vtk.Rendering.Core.vtkImageSlice;
const vtkImageMapper = vtk.Rendering.Core.vtkImageMapper;
const PiecewiseFun = vtk.Common.DataModel.vtkPiecewiseFunction.newInstance();
const ColorTransFun = vtk.Rendering.Core.vtkColorTransferFunction.newInstance();

container.style.display = 'flex';
container.style.position = 'static';
openglRenderWindow.setContainer(container);
const { width, height } = container.getBoundingClientRect();
openglRenderWindow.setSize(width, height);
renderWindow.addView(openglRenderWindow);

const imageActorI = vtkImageSlice.newInstance(); // 2D YZ images.
const imageActorJ = vtkImageSlice.newInstance(); // 2D XZ images.
const imageActorK = vtkImageSlice.newInstance(); // 2D XY images.
const imageMapperK = vtkImageMapper.newInstance();
const imageMapperJ = vtkImageMapper.newInstance();
const imageMapperI = vtkImageMapper.newInstance();

function getCenterOfScene (renderer) {
  const bounds = renderer.computeVisiblePropBounds();
  const center = [0, 0, 0];

  center[0] = (bounds[0] + bounds[1]) / 2.0;
  center[1] = (bounds[2] + bounds[3]) / 2.0;
  center[2] = (bounds[4] + bounds[5]) / 2.0;

  return center;
}

function updateColorLevel(e) {
        const colorLevel = Number((e ? e.target : document.querySelector('.colorLevel')).value);
        imageActorI.getProperty().setColorLevel(colorLevel);
        imageActorJ.getProperty().setColorLevel(colorLevel);
        imageActorK.getProperty().setColorLevel(colorLevel);
        renderWindow.render();
};

function updateColorWindow(e) {
        const colorLevel = Number((e ? e.target : document.querySelector('.colorWindow')).value);
        imageActorI.getProperty().setColorWindow(colorLevel);
        imageActorJ.getProperty().setColorWindow(colorLevel);
        imageActorK.getProperty().setColorWindow(colorLevel);
        renderWindow.render();
};

const viewColors = [
  [0.0, 0.0, 0.0], // sagittal
  [0.0, 0.0, 0.0], // coronal
  [0.0, 0.0, 0.0], // axial
];

function createRGBStringFromRGBValues(rgb) {
  if (rgb.length !== 3) {
    return 'rgb(0, 0, 0)';
  }
  return `rgb(${(rgb[0] * 255).toString()}, ${(rgb[1] * 255).toString()}, ${(rgb[2] * 255).toString()})`;
}

imageI = []
imageJ = []
imageK = []

  for (y = 0; y < tri_imageData.length; y++){

        const dataRange = tri_imageData[y].getPointData().getScalars().getRange();
        const extent = tri_imageData[y].getExtent();

        tri_imageData[y].setOrigin(0.0, 0.0, 0.0)

        image_origin = tri_imageData[y].getOrigin();
        console.log(image_origin)

        imageMapperK.setInputData(tri_imageData[y]);
        imageMapperK.setKSlice(0);
        imageActorK.setMapper(imageMapperK);

        imageMapperJ.setInputData(tri_imageData[y]);
        imageMapperJ.setJSlice(0);
        imageActorJ.setMapper(imageMapperJ);

        imageMapperI.setInputData(tri_imageData[y]);
        imageMapperI.setISlice(0);
        imageActorI.setMapper(imageMapperI);

        renderer.addActor(imageActorK);
        renderer.addActor(imageActorJ);
        renderer.addActor(imageActorI);

        ['.YZ', '.XZ', '.XY'].forEach((selector, idx) => {
                const el = document.querySelector(selector);
                el.setAttribute('min', extent[idx * 2 + 0]);
                el.setAttribute('max', extent[idx * 2 + 1]);
                el.setAttribute('value', 0);
        });

        ['.colorLevel', '.colorWindow'].forEach((selector) => {
                document.querySelector(selector).setAttribute('max', dataRange[1]);
                document.querySelector(selector).setAttribute('value', dataRange[1]);
        });

        document.querySelector('.colorLevel').setAttribute('value', (dataRange[0] + dataRange[1]) / 2);
        updateColorLevel();
        updateColorWindow();

        const axes = vtkAnnotatedCubeActor.newInstance();
        axes.setDefaultStyle({text: '+X', fontStyle: 'bold', fontFamily: 'Arial', fontColor: 'white',
        faceColor: createRGBStringFromRGBValues(viewColors[0]), edgeThickness: 0.1, edgeColor: 'white',
        resolution: 1600,
        });

        axes.setXMinusFaceProperty({text: '-X', faceColor: createRGBStringFromRGBValues(viewColors[0]),
        });

        axes.setYPlusFaceProperty({text: '+Y', faceColor: createRGBStringFromRGBValues(viewColors[1]),
        });

        axes.setYMinusFaceProperty({text: '-Y', faceColor: createRGBStringFromRGBValues(viewColors[1]),
        });

        axes.setZPlusFaceProperty({text: '+Z', faceColor: createRGBStringFromRGBValues(viewColors[2]),
        });

        axes.setZMinusFaceProperty({text: '-Z', faceColor: createRGBStringFromRGBValues(viewColors[2]),
        });

        // create orientation widget
        orientationWidget = vtkOrientationMarkerWidget.newInstance({actor: axes, interactor: interactor});
        setTimeout(() => {
        orientationWidget.setEnabled(true);
        orientationWidget.setViewportCorner(vtkOrientationMarkerWidget.Corners.BOTTOM_RIGHT);
        orientationWidget.setViewportSize(0.15);
        orientationWidget.setMinPixelSize(20);
        orientationWidget.setMaxPixelSize(80);
        }, 1);

        window.onresize = () => {orientationWidget.updateViewport();}

        imageI = imageActorI // 2D YZ images.
        imageJ = imageActorJ// 2D XZ images.
        imageK = imageActorK // 2D XY images.
  }

document.querySelector('.YZ').addEventListener('input', (e) => {
    imageActorI.getMapper().setISlice(Number(e.target.value));
    renderWindow.render();
});

document.querySelector('.XZ').addEventListener('input', (e) => {
    imageActorJ.getMapper().setJSlice(Number(e.target.value));
    renderWindow.render();
});

document.querySelector('.XY').addEventListener('input', (e) => {
    imageActorK.getMapper().setKSlice(Number(e.target.value));
    renderWindow.render();
});

document.querySelector('.colorLevel').addEventListener('input', updateColorLevel);
document.querySelector('.colorWindow').addEventListener('input', updateColorWindow);

var edge_K = edge_J = edge_I = true;

$("#XY_button").on('click', function() {
    imageK.setVisibility(edge_K);
    renderWindow.render();
    edge_K = !(edge_K);
});

$("#XZ_button").on('click', function() {
    imageJ.setVisibility(edge_J);
    renderWindow.render();
    edge_J = !(edge_J);
});

$("#YZ_button").on('click', function() {
    imageI.setVisibility(edge_I);
    renderWindow.render();
    edge_I = !(edge_I);
});

// const interactor = vtkRenderWindowInteractor.newInstance();
renderWindow.addRenderer(renderer);

interactor.setView(openglRenderWindow);
interactor.initialize();
interactor.bindEvents(container);

const center = getCenterOfScene(renderer);
const iStyle = vtkInteractorStyleManipulator.newInstance();

iStyle.setCenterOfRotation(center);

renderWindow.getInteractor().setInteractorStyle(iStyle);

renderer.resetCamera();
renderer.resetCameraClippingRange();
renderWindow.render()

}

I am too wondering why it’s creating two canvases.
Any suggestion regarding it?

And another question about .tif image.
C1-sample1_single.zip (3.0 MB)

Above I have uploaded a .tif image. When I open it in a glance-vtk viewer, the viewer can’t seem to render this file. The 3D viewer is blank at a glance.

I think it’s because of the alpha channel. And when I render it in my viewer it shows a black screen in composite mode and solid color in MaxIP like below.

Is it possible to ignore that channel while rendering it in 3D volume?

Thank you!

[Forrest]

For your first problem, are you calling your triplanar function multiple times? That could cause multiple canvases to appear.

For your second problem, you can set the component weights to zero. See the TestVolumeTypes example for more info on how to do that.

[Yuvi-416 (DMC-416-/)]

Hi there,

Thank you for your reply.
Yeah indeed there was some problem with an HTML page and I have fixed it now and now it’s rendering both images in one canvas. However, I lost control of the first image as control seemed fine for the second image. Moreover, I got the result however, it’s not what I thought.

I would like to show you what I wanted to do and I did that in ImageJ:

Steps:
First I load two images one by one and after that, I use the colour merge option to merge both images into one as shown below.

And the result I got while I load two images in a tri-planar viewer. You can see the slider only control the second image and I think even If I fix the control between these images, the image will overlap each other and I can’t see the background of the first or second image, because images might block the background or foreground of each other.

As I search whether it’s possible to do the same function on the web using ITK.js or Vtk.js (to merge colour instead of an image), however, I don’t seem to find an example or any hint about it.

Or maybe my concept is wrong. You can find the image below.
C1-sample1_single.zip (3.0 MB)
C2-sample1_single.zip (2.5 MB)

[Forrest]

vtk.js doesn’t have the color merging functionality. You will have to implement it yourself, or use itk-wasm to do this in C++ (cross-compiled to the web).