Can we merge a fMRI mask volume with a brain volume using vtk.js?

Web
1

I have a VTI brain volume that I display using vtkResliceCursor and a VTI volume containing voxel points of a fMRI mask,
I’m wondering is it possible with vtk.js to merge the two VTI files into one and display the brain volume with the fMRI mask like in the screenshot below:

image
The two white points of the fMRI mask are merged with the original volume of the brain

2

You can instantiate another pair of vtkImageReslice/vtkImageMapper/vtkImageSlice for your fMRI mask. You need to synchronize both vtkImageReslice instances (the one for your VTI brain volume and the one for your mask).
You will get some z-fighting issues that you can solve with ResolveCoincidentTopology parameters on your fmri actor.
You will also have to play with the LUT to make sure the fMRI is transparent so you can see behind (i.e. the resliced VTI brain volume)

3

Thank you for your helpful answer, by you saying that I need to synchronize both vtkImageReslice instances, I imagine that I must do like this:

const reader = vtkXMLImageDataReader.newInstance({ fetchGzip: true });
reader.setUrl("/Patient/volume.vti").then(() => {
 reader.loadData().then(() => {
  widget.setImage(reader.getOutputData())
  const reader1 = vtkXMLImageDataReader.newInstance({ fetchGzip: true });
   reader1.setUrl("/Patient/fMRImask.vti").then(() => {
     reader1.loadData().then(() => {
      widget.setImage(reader1.getOutputData(););
  ...

Or is there something existing to do that?

4

I meant more something like that (pseudo code):

// Create RCW on brain image only
widget.setImage(brainReader.getOutput());
...
// 
fmriImageReslice= vtkImageReslice.newInstance(..);
fmriImageReslice.setInputData(fmri);
fmriImageResliceMapper = vtkImageMapper.newInstance(..);
...
brainWidgetView.onInteractionEvent() {
updateReslice(..., brainReslice, ...)
updateReslice(..., fmriReslice, ...)
}
...
6

Thank you, I did what you told me (I guess) but the mask was not displayed, and if I delete the brain actor from the scene I don’t see the fMRI mask (I see it if I do only onz widget.setImage(fMRImask) in clear shiny white (alone without the brain of course)),

PS: I added the brainActor after adding the fMRI mask actor to solve that z-index issue you told me,

obj.renderer.addActor(obj.resliceActorfMRI);
obj.renderer.addActor(obj.resliceActor);

I shrinked some parts of code:

for (let i = 0; i < 3; i++) {
 const obj = {
    ... 
 };
  obj.widgetManager.setCaptureOn(CaptureOn.MOUSE_MOVE);
  // for brain VTI
  obj.reslice = vtkImageReslice.newInstance();
  obj.reslice.setSlabMode(SlabMode.MEAN);
  ...
  obj.reslice.setOutputDimensionality(2);

  obj.resliceMapper = vtkImageMapper.newInstance();
  obj.resliceMapper.setInputConnection(obj.reslice.getOutputPort());
  obj.resliceActor = vtkImageSlice.newInstance();
  obj.resliceActor.setMapper(obj.resliceMapper);
  // for fMRI mask VTI PS: it is not shrinked 
  obj.reslicefMRI = vtkImageReslice.newInstance();
  obj.resliceMapperfMRI = vtkImageMapper.newInstance();
  obj.resliceMapperfMRI .setInputConnection(obj.reslicefMRI .getOutputPort());
  obj.resliceActorfMRI = vtkImageSlice.newInstance();
  obj.resliceActorfMRI .setMapper(obj.resliceMapperfMRI );

const reader = vtkXMLImageDataReader.newInstance({ fetchGzip: true });
//Reading the brain volume
reader.setUrl("brainVolume.vti").then(() => {
  reader.loadData().then(() => {
   const imageBrain = reader.getOutputData();
    widget.setImage(imageBrain);
    const reader2 = vtkXMLImageDataReader.newInstance({ fetchGzip: true });
    //reading the VTI of the mask
    reader2.setUrl("fMRImask.vti").then(() => {
      reader2.loadData().then(() => {
      const imagefMRI = reader2.getOutputData();

      const outline = vtkOutlineFilter.newInstance();
      ...
      outlineActor.setMapper(outlineMapper);

      viewAttributes.forEach((obj, i) => {
        obj.reslice.setInputData(imageBrain);
        obj.reslicefMRI.setInputData(imagefMRI);

        obj.renderer.addActor(obj.resliceActorfMRI);
        obj.renderer.addActor(obj.resliceActor);

        const viewType = xyzToViewType[i];
        viewAttributes.forEach((v) => {
            v.widgetInstance.onInteractionEvent(
              (...) => {
                const activeViewType = ...;
                const keepFocalPointPosition = ...;
                updateReslice({
                  viewType: viewType,
                  reslice: obj.reslice,
                  actor: obj.resliceActor,
                  renderer: obj.renderer,
                  resetFocalPoint: false,
                  keepFocalPointPosition,
                  computeFocalPointOffset,
                  sphereSources: obj.sphereSources,
                });

                updateReslice({
                  viewType: viewType,
                  reslice: obj.reslicefMRI,
                  actor: obj.resliceActorfMRI,
                  renderer: obj.renderer,
                  resetFocalPoint: false,
                  keepFocalPointPosition,
                  computeFocalPointOffset,
                  sphereSources: obj.sphereSources,
                });
              }
            );
          });

        updateReslice({
          viewType,
          reslice,
          actor: obj.resliceActor,
          renderer: obj.renderer,
          resetFocalPoint: true, 
          keepFocalPointPosition: false,
          computeFocalPointOffset: true, 
          sphereSources: obj.sphereSources,
        });
        obj.renderWindow.render();
      });
...
7

That’s the idea I guess.

Just make sure you did not forget obj.reslicefMRI.setOutputDimensionality(2); or other reslice properties.

I don’t have much more ideas without trying to reproduce and digging into the details (got funding ? :slight_smile: ).

1 Like
9

Thank you for your precious help, for now my company and I are just experimenting what we can do with vtk.js, for sure I will talk to them