How can I visualize passed series of Dicom inputs in ResliceCursorWidget?

Web
1

Hi everyone,

I am trying to implement ResliceCursorWidget (vtk.js) in my project. I was able to run to the example vtk.js) without a problem, but when I change an input file I am getting some errors like Invalid or Missing Input or No Input but it still shows X+ axes but not others.

As I have a series of Dicom slices and I am reading it with ITK and using the ITK-TO-VTK function for converting it into VTK format and then passing those arrays from : for (i = 0; i < imageData.length; i++){}

I couldn’t understand which parameter should I change so that all planes can show the image and as well as it can generate a 3D view.

As I can visualize my input without a problem while I use ResliceCursor implementation.

///////////// Functions /////////////////////////////////
//////////////////////////////////////////////////////////
function volume(imageData){

const SlabMode = {
  MIN: 0,
  MAX: 1,
  MEAN: 2,
  SUM: 3,
};

const ViewTypes = {
  DEFAULT: 0,
  GEOMETRY: 1,
  SLICE: 2,
  VOLUME: 3,
  YZ_PLANE: 4, // Sagittal
  XZ_PLANE: 5, // Coronal
  XY_PLANE: 6, // Axial
};

const CaptureOn = {
  MOUSE_MOVE: 0,
  MOUSE_RELEASE: 1,
};

const xyzToViewType = [
  ViewTypes.YZ_PLANE,
  ViewTypes.XZ_PLANE,
  ViewTypes.XY_PLANE,
];
*emphasized text*
//////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////
/////  CALL THE VTK FUNCTIONS ///////////////////////////////////
////////////////////////////////////////////////////////////////
const vtkPiecewiseFunction = vtk.Common.DataModel.vtkPiecewiseFunction;
const vtkColorTransferFunction = vtk.Rendering.Core.vtkColorTransferFunction;
const vtkVolume = vtk.Rendering.Core.vtkVolume;
const vtkVolumeMapper = vtk.Rendering.Core.vtkVolumeMapper;
const vtkImageSlice = vtk.Rendering.Core.vtkImageSlice;
const vtkInteractorStyleTrackballCamera = vtk.Interaction.Style.vtkInteractorStyleTrackballCamera;
const vtkImageMapper = vtk.Rendering.Core.vtkImageMapper;
const vtkInteractorStyleImage = vtk.Interaction.Style.vtkInteractorStyleImage;
const vtkActor = vtk.Rendering.Core.vtkActor;
const vtkAnnotatedCubeActor = vtk.Rendering.Core.vtkAnnotatedCubeActor;
const vtkDataArray = vtk.Common.Core.vtkDataArray;
const vtkHttpDataSetReader = vtk.IO.Core.vtkHttpDataSetReader;
const vtkGenericRenderWindow = vtk.Rendering.Misc.vtkGenericRenderWindow;
// const vtkImageData = vtk.Common.DataModel.vtkImageData;
const vtkImageReslice = vtk.Imaging.Core.vtkImageReslice;
const vtkMapper = vtk.Rendering.Core.vtkMapper;
const vtkOutlineFilter = vtk.Filters.General.vtkOutlineFilter;
const vtkOrientationMarkerWidget = vtk.Interaction.Widgets.vtkOrientationMarkerWidget;
const vtkResliceCursorWidget = vtk.Widgets.Widgets3D.vtkResliceCursorWidget;
const vtkWidgetManager = vtk.Widgets.Core.vtkWidgetManager;
const vtkSphereSource = vtk.Filters.Sources.vtkSphereSource;
const controlPanel = vtkResliceCursorWidget.controlPanel;
// const { vec3 } = 'gl-matrix';


///////////////////////////////////////////////////////////////////////////////////
// ----------------------------------------------------------------------------
// Define main attributes
// ----------------------------------------------------------------------------

const viewColors = [
  [0.5, 0.5, 0.5], // sagittal
  [0.5, 0.5, 0.5], // coronal
  [0.5, 0.5, 0.5], // axial
  [0.5, 0.5, 0.5], // 3D
];

const viewAttributes = [];
const widget = vtkResliceCursorWidget.newInstance();
const widgetState = widget.getWidgetState();
widgetState.setKeepOrthogonality(true);
widgetState.setOpacity(0.6);
widgetState.setSphereRadius(10);
widgetState.setLineThickness(5);

const showDebugActors = true;
///////////////////////////////////////////////////////////////////////////////////
// ----------------------------------------------------------------------------
// Define html structure
// ----------------------------------------------------------------------------

const container = document.querySelector('body');
const controlContainer = document.createElement('div');
controlContainer.innerHTML = controlPanel;
container.appendChild(controlContainer);

///////////////////////////////////////////////////////////////////////////////////
// ----------------------------------------------------------------------------
// Setup rendering code
// ----------------------------------------------------------------------------
/**
 * Function to create synthetic image data with correct dimensions
 * Can be use for debug
 * @param {Array[Int]} dims
 */
// eslint-disable-next-line no-unused-vars
//////////////////////////////////////////////////////////////////
//function createSyntheticImageData(dims) {
//  const imageData = vtkImageData.newInstance();
//  const newArray = new Uint8Array(dims[0] * dims[1] * dims[2]);
//  const s = 0.1;
//  imageData.setSpacing(s, s, s);
//  imageData.setExtent(0, 127, 0, 127, 0, 127);
//  let i = 0;
//  for (let z = 0; z < dims[2]; z++) {
//    for (let y = 0; y < dims[1]; y++) {
//      for (let x = 0; x < dims[0]; x++) {
//        newArray[i++] = (256 * (i % (dims[0] * dims[1]))) / (dims[0] * dims[1]);
//      }
//    }
//  }
//
//  const da = vtkDataArray.newInstance({ numberOfComponents: 1, values: newArray, });
//  da.setName('scalars');
//
//  imageData.getPointData().setScalars(da);
//
//  return imageData;
//}
////////////////////////////////////////////////////////////////////////////////////
function createRGBStringFromRGBValues(rgb) {
  if (rgb.length !== 3) {
    return 'rgb(0, 0, 0)';
  }
  return `rgb(${(rgb[0] * 255).toString()}, ${(rgb[1] * 255).toString()}, ${(
    rgb[2] * 255
  ).toString()})`;
}

const initialPlanesState = { ...widgetState.getPlanes() };

let view3D = null;
///////////////////////////////////////////////////////////////////////////////////
/////// create axes, orientation widget
for (let i = 0; i < 4; i++) {
  const element = document.createElement('div');
  element.setAttribute('class', 'view');
  element.style.width = '50%';
  element.style.height = '300px';
  element.style.display = 'inline-block';
  container.appendChild(element);

  const grw = vtkGenericRenderWindow.newInstance();
  grw.setContainer(element);
  grw.resize();
  const obj = {
    renderWindow: grw.getRenderWindow(),
    renderer: grw.getRenderer(),
    GLWindow: grw.getOpenGLRenderWindow(),
    interactor: grw.getInteractor(),
    widgetManager: vtkWidgetManager.newInstance(),
  };

  obj.renderer.getActiveCamera().setParallelProjection(true);
  obj.renderer.setBackground(...viewColors[i]);
  obj.renderWindow.addRenderer(obj.renderer);
  obj.renderWindow.addView(obj.GLWindow);
  obj.renderWindow.setInteractor(obj.interactor);
  obj.interactor.setView(obj.GLWindow);
  obj.interactor.initialize();
  obj.interactor.bindEvents(element);
  obj.widgetManager.setRenderer(obj.renderer);
  if (i < 3) {
    obj.interactor.setInteractorStyle(vtkInteractorStyleImage.newInstance());
    obj.widgetInstance = obj.widgetManager.addWidget(widget, xyzToViewType[i]);
    obj.widgetInstance.setScaleInPixels(true);
    obj.widgetInstance.setRotationHandlePosition(0.75); //////////////////////////////////
    obj.widgetManager.enablePicking();
    // Use to update all renderers buffer when actors are moved
    obj.widgetManager.setCaptureOn(CaptureOn.MOUSE_MOVE);
  } else {
    obj.interactor.setInteractorStyle(
      vtkInteractorStyleTrackballCamera.newInstance()
    );
  }

  obj.reslice = vtkImageReslice.newInstance();
  // obj.reslice.setSlabMode(SlabMode.MEAN);
  // obj.reslice.setSlabNumberOfSlices(1);
  obj.reslice.setTransformInputSampling(false);
  obj.reslice.setAutoCropOutput(true);
  obj.reslice.setOutputDimensionality(2); //////////////////////////////
  obj.resliceMapper = vtkImageMapper.newInstance();
  obj.resliceMapper.setInputConnection(obj.reslice.getOutputPort());
  obj.resliceActor = vtkImageSlice.newInstance();
  obj.resliceActor.setMapper(obj.resliceMapper);
  obj.sphereActors = [];
  obj.sphereSources = [];

  // Create sphere for each 2D views which will be displayed in 3D
  // Define origin, point1 and point2 of the plane used to reslice the volume
  for (let j = 0; j < 3; j++) {
    const sphere = vtkSphereSource.newInstance();
    sphere.setRadius(10); /////////////////////////////////////////////////////////
    const mapper = vtkMapper.newInstance();
    mapper.setInputConnection(sphere.getOutputPort());
    const actor = vtkActor.newInstance();
    actor.setMapper(mapper);
    actor.getProperty().setColor(...viewColors[i]);
    actor.setVisibility(showDebugActors);
    obj.sphereActors.push(actor);
    obj.sphereSources.push(sphere);
  }

  if (i < 3) {
    viewAttributes.push(obj);
  } else {
    view3D = obj;
  }

  // create axes ////////////////////////////////////////////////////////////////////////////////////////////
  const axes = vtkAnnotatedCubeActor.newInstance();
  axes.setDefaultStyle({
    text: '+X',
    fontStyle: 'bold',
    fontFamily: 'Arial',
    fontColor: 'black',
    fontSizeScale: (res) => res / 2,
    faceColor: createRGBStringFromRGBValues(viewColors[0]),
    faceRotation: 0,
    edgeThickness: 0.1,
    edgeColor: 'black',
    resolution: 400,
  });
  // axes.setXPlusFaceProperty({ text: '+X' });
  axes.setXMinusFaceProperty({
    text: '-X',
    faceColor: createRGBStringFromRGBValues(viewColors[0]),
    faceRotation: 90,
    fontStyle: 'italic',
  });
  axes.setYPlusFaceProperty({
    text: '+Y',
    faceColor: createRGBStringFromRGBValues(viewColors[1]),
    fontSizeScale: (res) => res / 4,
  });
  axes.setYMinusFaceProperty({
    text: '-Y',
    faceColor: createRGBStringFromRGBValues(viewColors[1]),
    fontColor: 'white',
  });
  axes.setZPlusFaceProperty({
    text: '+Z',
    faceColor: createRGBStringFromRGBValues(viewColors[2]),
  });
  axes.setZMinusFaceProperty({
    text: '-Z',
    faceColor: createRGBStringFromRGBValues(viewColors[2]),
    faceRotation: 45,
  });
////////////////////////////////////////////////////////////////////////////////////////////////////////////////
  // create orientation widget
  const orientationWidget = vtkOrientationMarkerWidget.newInstance({
    actor: axes,
    interactor: obj.renderWindow.getInteractor(),
  });
  orientationWidget.setEnabled(true);
  orientationWidget.setViewportCorner(vtkOrientationMarkerWidget.Corners.BOTTOM_RIGHT);
  orientationWidget.setViewportSize(0.15);
  orientationWidget.setMinPixelSize(100);
  orientationWidget.setMaxPixelSize(300);
}
///////////////////////////////////////////////////////////////////////////////////
// ----------------------------------------------------------------------------
// Load image
// ----------------------------------------------------------------------------
function updateReslice(interactionContext = {
    viewType: '', reslice: null, actor: null, renderer: null,resetFocalPoint: false, keepFocalPointPosition: false,
    computeFocalPointOffset: false, spheres: null,
})
{
  const obj = widget.updateReslicePlane( interactionContext.reslice, interactionContext.viewType);
  if (obj.modified) {
    // Get returned modified from setter to know if we have to render
    interactionContext.actor.setUserMatrix(interactionContext.reslice.getResliceAxes());
    interactionContext.sphereSources[0].setCenter(...obj.origin);
    interactionContext.sphereSources[1].setCenter(...obj.point1);
    interactionContext.sphereSources[2].setCenter(...obj.point2);
  }
  widget.updateCameraPoints(
    interactionContext.renderer, interactionContext.viewType, interactionContext.resetFocalPoint,
    interactionContext.keepFocalPointPosition, interactionContext.computeFocalPointOffset );
  view3D.renderWindow.render();
  return obj.modified;
}
//////////////////////////////////////////////////////////////////////////////////
///// const PATH ='https://kitware.github.io/vtk-js/'
// LIDC2.vti
// headsq.vti

// const reader = vtkHttpDataSetReader.newInstance({fetchGzip: true, });

// reader.setUrl(`${PATH}/data/volume/LIDC2.vti`).then(() => {
//     reader.loadData().then(() => {
   for (i = 0; i < imageData.length; i++){

        // const image = imageData[i].getOutputData();
        // console.log(image);
        // const image = imageData[i].getPointData().getScalars().getRange();
        // console.log(image);
        widget.setImage(imageData[i]); //////////////////////////////////////////////
        console.log(widget);

        // Create image outline in 3D view
        const outline = vtkOutlineFilter.newInstance();
        outline.setInputData(imageData[i]); ////////////////////////////////////////////
        const outlineMapper = vtkMapper.newInstance();
        outlineMapper.setInputData(outline.getOutputData());
        const outlineActor = vtkActor.newInstance();
        outlineActor.setMapper(outlineMapper);
        view3D.renderer.addActor(outlineActor);

        viewAttributes.forEach((obj, i) => {
            obj.reslice.setInputData(imageData[i]); ////////////////////////////////////////////
            obj.renderer.addActor(obj.resliceActor);
            view3D.renderer.addActor(obj.resliceActor);
            obj.sphereActors.forEach((actor) => {
                obj.renderer.addActor(actor);
                view3D.renderer.addActor(actor);
            });
            const reslice = obj.reslice;
            const viewType = xyzToViewType[i];

            viewAttributes.forEach((v) => {
                v.widgetInstance.onInteractionEvent(
                ({ computeFocalPointOffset, canUpdateFocalPoint }) => {
                      const activeViewType = widget.getWidgetState().getActiveViewType();
                      const keepFocalPointPosition = activeViewType !== viewType && canUpdateFocalPoint;
                      updateReslice({viewType, reslice, actor: obj.resliceActor, renderer: obj.renderer, resetFocalPoint: false,
                      keepFocalPointPosition, computeFocalPointOffset, sphereSources: obj.sphereSources, });
                }
              );
            });
            updateReslice({ viewType, reslice, actor: obj.resliceActor, renderer: obj.renderer,
                resetFocalPoint: true,keepFocalPointPosition: false,computeFocalPointOffset: true,
                sphereSources: obj.sphereSources,
            });
            obj.renderWindow.render();
        });
    view3D.renderer.resetCamera();
    view3D.renderer.resetCameraClippingRange();
   }

//    dimensions = imageData[i].getDimensions()
//    console.log(dimensions);
// set max number of slices to slider.
//    const maxNumberOfSlices = vec3.length(imageData[i].getDimensions());
//    console.log(maxNumberOfSlices);
//    document.getElementById('slabNumber').max = maxNumberOfSlices;
//  });

///////////////////////////////////////////////////////////////////////////////////
// ----------------------------------------------------------------------------
// Define panel interactions
// ----------------------------------------------------------------------------
function updateViews() {
  viewAttributes.forEach((obj, i) => {
    updateReslice({ viewType: xyzToViewType[i], reslice: obj.reslice, actor: obj.resliceActor, renderer: obj.renderer,
      resetFocalPoint: true, keepFocalPointPosition: false, computeFocalPointOffset: true,
      sphereSources: obj.sphereSources, resetViewUp: true,
    });
    obj.renderWindow.render();
  });
  view3D.renderer.resetCamera();
  view3D.renderer.resetCameraClippingRange();
}

//const checkboxOrthogonality = document.getElementById('checkboxOrthogality');
//checkboxOrthogonality.addEventListener('change', (ev) => {
//  widgetState.setKeepOrthogonality(checkboxOrthogonality.checked);
//});
//
//const checkboxRotation = document.getElementById('checkboxRotation');
//checkboxRotation.addEventListener('change', (ev) => {
//  widgetState.setEnableRotation(checkboxRotation.checked);
//});
//
//const checkboxTranslation = document.getElementById('checkboxTranslation');
//checkboxTranslation.addEventListener('change', (ev) => {
//  widgetState.setEnableTranslation(checkboxTranslation.checked);
//});
//
//const checkboxScaleInPixels = document.getElementById('checkboxScaleInPixels');
//checkboxScaleInPixels.addEventListener('change', (ev) => {
//  viewAttributes.forEach((obj) => {
//    obj.widgetInstance.setScaleInPixels(checkboxScaleInPixels.checked);
//    obj.interactor.render();
//  });
//});
/////////////////////////////////////////////////////////////////////////////////////
//const optionSlabModeMin = document.getElementById('slabModeMin');
//optionSlabModeMin.value = SlabMode.MIN;
//const optionSlabModeMax = document.getElementById('slabModeMax');
//optionSlabModeMax.value = SlabMode.MAX;
//const optionSlabModeMean = document.getElementById('slabModeMean');
//optionSlabModeMean.value = SlabMode.MEAN;
//const optionSlabModeSum = document.getElementById('slabModeSum');
//optionSlabModeSum.value = SlabMode.SUM;
//const selectSlabMode = document.getElementById('slabMode');
//selectSlabMode.addEventListener('change', (ev) => {
//  viewAttributes.forEach((obj) => {
//    obj.reslice.setSlabMode(Number(ev.target.value));
//  });
//  updateViews();
//});
//
//const sliderSlabNumberofSlices = document.getElementById('slabNumber');
//sliderSlabNumberofSlices.addEventListener('change', (ev) => {
//  const trSlabNumberValue = document.getElementById('slabNumberValue');
//  trSlabNumberValue.innerHTML = ev.target.value;
//  viewAttributes.forEach((obj) => {
//    obj.reslice.setSlabNumberOfSlices(ev.target.value);
//  });
//  updateViews();
//});
//
//const buttonReset = document.getElementById('buttonReset');
//buttonReset.addEventListener('click', () => {
//  widgetState.setPlanes({ ...initialPlanesState });
//  widget.setCenter(widget.getWidgetState().getImage().getCenter());
//  updateViews();
//});

}

ERROR:

Screenshot from 2022-03-24 14-31-27
Screenshot from 2022-03-24 14-31-271872×831 134 KB

How can I visualize passed series of Dicom inputs beside LIDC2.vti and headsq.vti in ResliceCursorWidget?

Thank you

2

I’m guessing that “Invalid or missing input” is being reported by vtkOutlineFilter, and that “no input” is being reported by the ImageMapper. Double check that there is data being passed to those filters.

3

Thank you for the suggestion.
There was a problem of notation in for loop.
After I change the for loop notation it worked.

I am feeling some lag while I slide the slider. Is it normal? or Is there a way to make it faster.

4

What sort of lag? Can you provide a small video for that?

5

I think it was bcoz of the image size. When I view a small number of slices it works perfectly.

6

If it’s due to ImageReslice, then it’ll be slow for large images. That’s done on the CPU. GPU-based reslicing involves reslicing a volume with a slab, which is achievable via the VolumeMapper cropping planes.

7

Can you provide one example based on this one?

And one thing more, today I try to visualize a 16-bit image but the whole render screen got white like below.

Screenshot from 2022-03-31 16-15-30
Screenshot from 2022-03-31 16-15-30881×937 17.7 KB

I tried to search how can I increase the colour-bit but I couldn’t find the solution.
Is it related to vtkImageSlice?

8

I think it was a problem with colorLevel and colorWindow.
But I couldn’t find a way to manipulate it.

As 16bit image dataRange= imageData[i].getPointData().getScalars().getRange(); which is (353, 65523):

It’s really very hard to slide the level/window inside the MPR. Is there a better way to adjust it?

Thank you.

9

Huh, does the image show up as all white in Glance? If so, might be a rendering issue.

I would suggest implementing sliders that update the mapper’s colorLevel and colorWindow.

10

I checked it with a glance and I am impressed how a glance controls the window/level of all three at once.

How can I achieve that? I think it will be helpful rather than maintaining window/level for all screens one by one.

Screenshot from 2022-03-31 21-30-28
Screenshot from 2022-03-31 21-30-281844×946 114 KB

And I was also wondering what would be the way to fix the view which only shows in one way. like below:

Screenshot from 2022-03-31 21-37-39
Screenshot from 2022-03-31 21-37-391856×879 71.7 KB