Brain segmentation on .VTK head surface


I have a VTK surface of a human head (image below):

I need to apply peeling to it to have the shape of the brain, (segmentation of the brain) using coordinates from the VTI volume data (MRI of the head),

So using the right depth from the VTI volume, How can I peel the VTK head to have the shape of the brain?

PS: In C++ I have an algorithm where they managed to do this using vtkProbeFilter, however we can’t use vtkProbeFilter in vtk.js

to finally have the brain

1 Like

I have forgotten to precise, we don’t suppose that we know where the depth of the brain is, we only peel (using a slider) until we can see satisfying results, so using the depth value (from the slider), and the VTI volume (MRI of the head) coordinates, we peel the VTK skin head to be identical to the VTI volume at certain depth

Can you please explain the VTK pipeline you have in C++ to do the segmentation? I can hardly see how vtkProbeFilter can be used to do segmentation.

Hey Julien,
I’m sorry for the late ressponse, sorry I didn’t want to say “by segmentation” the real segmentation that we use in the medical field, It’s just a reshaping of the VTK surface to be identic to the VTI volume at a certain depth, like the video there,

For the pipeline (C++) to do so:

  • We charge a .nii MRI and a Skin VTK (a human head)
  • We get the bounds (bounds[6]) from the nii reader.
  • We calculate sizes:
    xSize = bounds[1] - bounds[0];
    ySize = bounds[3] - bounds[2];
    zSize = bounds[5] - bounds[4];
  • We create a vtkPlane that we set its Origin values to Sizes
    vtkSmartPointer<vtkPlane> plane = vtkSmartPointer<vtkPlane>::New();
    plane->SetNormal(0.0, 0.0, 1.0);
  • We create a vtkDecimatePro that we set Input Connection to the Output of skin VTK reader
  vtkSmartPointer<vtkDecimatePro> deci=vtkSmartPointer<vtkDecimatePro>::New();
  • We create a vtkSmoothPolyDataFilter that we set its Input Connection to vtkDecimatePro Outport
  • We create a vtkTransform that we set its Translate to Sizes/2
  vtkSmartPointer<vtkTransform> transform = vtkSmartPointer<vtkTransform>::New();
  • We create vtkTransformPolyDataFilter that we set its Input Connection to vtkSmoothPolyDataFilter and its Transform to vtkTransform:
  vtkSmartPointer<vtkTransformPolyDataFilter> transformFilter = vtkSmartPointer<vtkTransformPolyDataFilter>::New();

  • Here where we use vtkProbeFilter :
  vtkSmartPointer<vtkProbeFilter> probe = vtkSmartPointer<vtkProbeFilter>::New();
  vtkSmartPointer<vtkPolyDataMapper> map = vtkSmartPointer<vtkPolyDataMapper>::New();
  vtkSmartPointer<vtkActor> actor = vtkSmartPointer<vtkActor>::New();
1 Like

Understood, you have a vtkImageData (.vti or nii) AND a vtkPolyData (.vtp or vtk). That’s what I was missing.

You should be able to do the same in VTK.js.

To replace vtkProbeFilter, you can simply create a point data array (where a scalar is associated at each point), and you manually browse each point (xyz coordinate), convert the coordinate into ijk (i = (x - origin[0]) / scaling[0], j = (y - origin[1]) / scaling[1]…) and get the voxel value at this local coordinate.

1 Like

Thank you Julien I’m going to do this

Hey Julien,

I have managed browse each point and convert the coordinate, and get the voxel value at this local coordinate,

The next step is directly apply a transformation filter to my vtkPolyData?

I have managed browse each point and convert the coordinate, and get the voxel value at this local coordinate,

Great so you have replaced vtkProbeFilter (if you correctly added the voxel value into a vtkDataArray( e.g. vtkDataArray.newInstance({name: 'scalars', values: new UInt16Array(scalarValuesHere), numberOfComponents: 1});

The next step is directly apply a transformation filter to my vtkPolyData?

You can use vtkMatrixBuilder.buildFromDegree().scale(...).apply(polyData.getPoints().getData()) to apply a transform on ALL the vtkPolyData points.

1 Like

Thank you for your answer,

But vtkMatrixBuilder.buildFromDegree().scale(...).apply(polyData.getPoints().getData()) the scale function, only takes 3 parameters (sx,sy,sz), where I can use my vtkDataArray that has scalars values?

vtkMatrixBuilder is for geometrically transforming 3D points (x, y ,z) and not scalar values.
I do not know what that means to “transform” scalar values.

I’m sorry I didn’t correctly explain my question,
This is what I did,

const bounds = reader.VTIgetOutputData().getBounds();
    const scaleParam = 0.5

    const xSize = bounds[1] - bounds[0];
    const ySize = bounds[3] - bounds[2];
    const zSize = bounds[5] - bounds[4]

    const image = readerVTI.getOutputData();
    const polydata = readerVTK.getOutputData();

    translate(-xSize/2, -ySize/2, -zSize/2).
    translate(xSize/2, ySize/2, zSize/2).

// Creating a new DataArray that will store voxels (In vtk.js I didn't find setNumberOfTuples() like python and C++, but I continued without)
   const colors = vtkDataArray.newInstance({values: new Uint8Array(polydata.getNumberOfCells()), numberOfComponents:3});


//Manually browse all the mesh cells, get its coordinates and get the corresponding voxel value in the volume
    for (var currentCellIndex=0; currentCellIndex<polydata.getNumberOfCells(); currentCellIndex++){
      var currentCell = polydata.getCell(currentCellIndex)
      const currentPoints = currentCell.getPoints()
      var currentcellX = 0
      var currentcellY = 0
      var currentcellZ = 0

      const currentCellCoordinate = [
          Math.floor(currentcellX / currentPoints.getNumberOfPoints()),
          Math.floor(currentcellY / currentPoints.getNumberOfPoints()),
          Math.floor(currentcellZ / currentPoints.getNumberOfPoints())
      //'Color cell'
      const currentVxValue = image.getScalarValueFromWorld(currentCellCoordinate[0], currentCellCoordinate[1], currentCellCoordinate[2],0)
// I guess the problem is here because I'm not getting the right voxels values
      colors.setTuple(currentCellIndex, [currentVxValue, currentVxValue, currentVxValue])

    renderer.setBackground([.3, .2, .1]);

In the end I’m getting like a 1/3 of the head with no colors:

I wouldn’t browse cells but only points. I also wouldn’t create an RGB scalar array, but a simple scalar array:

const points = polyData.getPoints();
const scalars = vtkDataArray.newInstance({dataType: image.getPointData().getDataType(), size:  points.getNumberOfPoints(), numberOfComponents:1});

const xyz = [];
for (let i = 0; i <  points.getNumberOfPoints(); ++i) {
  points.getPoint(i, xyz);
  const currentVxValue = image.getScalarValueFromWorld(xyz,0);
  scalars.setTuple(i, [currentVxValue]);

Thank you, yes what you wrote is more simple and better, image.getPointData().getDataType() is not a function in vtk.js, it doesn’t work, I didn’t use it.
The result that I’ve obtained is better than the previous, but I don’t get the total of the head


1 Like

Yes exactly, I forgot to edit it, thank you so much,
I don’t know how to solve the problems of Extent (you can see in the CodeSandBox),
my volume and polydata have not the same Bounds values

GetScalarPointer: Pixel 113.70,159.73,-5.47 is not in memory. Current extent = 0,181,0,255,0,255

I tried to do something like this :
but it seems that is not the right solution.

You need to invert the matrix multiplication orders:

              .translate(xSize / 2, ySize / 2, zSize / 2)
              .scale(scaleParam, scaleParam, scaleParam)
              .translate(-xSize / 2, -ySize / 2, -zSize / 2)

Thank you so much it worked, that was really helpful