# Brain segmentation on .VTK head surface

**URL:** https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706
**Category:** Web
**Created:** [October 31, 2022, 9:41am UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706 "2022-10-31T09:41:27Z")
**Posts on this page:** 17
**Page:** 1

<div class="post-metadata">

### Author: ![peeling](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/p/b5e925/32.png) [@peeling](https://discourse.vtk.org/u/peeling)
#### Post date: [October 31, 2022, 9:41am UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/1 "2022-10-31T09:41:27Z")

</div>

Hey,

I have a VTK surface of a human head (image below):

 ![image](https://discourse.vtk.org/uploads/default/original/2X/6/67ccf25c71b1043d60944496a9c8c88c6f344908.jpeg)

I need to apply peeling to it to have the shape of the brain, (segmentation of the brain) using coordinates from the VTI volume data (MRI of the head),

So using the right depth from the VTI volume, How can I peel the VTK head to have the shape of the brain?

 ![image](https://discourse.vtk.org/uploads/default/original/2X/a/a33082cdd1e5b8a92aa59a346b242f8634614e02.jpeg)

PS: In C++ I have an algorithm where they managed to do this using [vtkProbeFilter](http://www.geologie.uni-freiburg.de/root/manuals/vtkman/vtkProbeFilter.html), however we can’t use vtkProbeFilter in vtk.js

to finally have the brain

 ![image](https://discourse.vtk.org/uploads/default/original/2X/f/f89c528ac388c1cda09d6bdcc2ffa3a4af63c88f.jpeg)

---

<div class="post-metadata">

### Author: ![peeling](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/p/b5e925/32.png) [@peeling](https://discourse.vtk.org/u/peeling)
#### Post date: [November 2, 2022, 8:47am UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/2 "2022-11-02T08:47:34Z")

</div>

I have forgotten to precise, we don’t suppose that we know where the depth of the brain is, we only peel (using a slider) until we can see satisfying results, so using the depth value (from the slider), and the VTI volume (MRI of the head) coordinates, we peel the VTK skin head to be identical to the VTI volume at certain depth

---

<div class="post-metadata">

### Author: ![finetjul](https://discourse.vtk.org/user_avatar/discourse.vtk.org/finetjul/32/154_2.png) [@finetjul](https://discourse.vtk.org/u/finetjul)
#### Post date: [November 2, 2022, 11:05am UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/3 "2022-11-02T11:05:15Z")

</div>

Can you please explain the VTK pipeline you have in C++ to do the segmentation? I can hardly see how vtkProbeFilter can be used to do segmentation.

---

<div class="post-metadata">

### Author: ![peeling](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/p/b5e925/32.png) [@peeling](https://discourse.vtk.org/u/peeling)
#### Post date: [November 10, 2022, 11:28am UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/4 "2022-11-10T11:28:09Z")

</div>

Hey Julien,  
I’m sorry for the late ressponse, sorry I didn’t want to say “by segmentation” the real segmentation that we use in the medical field, It’s just a reshaping of the VTK surface to be identic to the VTI volume at a certain depth, like the video there,  
 ![ezgif-1-144f08ef9f](https://discourse.vtk.org/uploads/default/original/2X/2/2216281991c0340bedcc0578d24cfcc8deab6df7.gif)

For the pipeline (C++) to do so:

- We charge a .nii MRI and a Skin VTK (a human head)
- We get the bounds (bounds[6]) from the nii reader.
- We calculate sizes:

```auto
    xSize = bounds[1] - bounds[0];
    ySize = bounds[3] - bounds[2];
    zSize = bounds[5] - bounds[4];

```

- We create a vtkPlane that we set its Origin values to Sizes

```auto
    vtkSmartPointer<vtkPlane> plane = vtkSmartPointer<vtkPlane>::New();
    plane->SetOrigin(xSize/2.0,ySize/2.0,zSize/2.0);
    plane->SetNormal(0.0, 0.0, 1.0);

```

- We create a vtkDecimatePro that we set Input Connection to the Output of skin VTK reader

```auto
  vtkSmartPointer<vtkDecimatePro> deci=vtkSmartPointer<vtkDecimatePro>::New();
  deci->SetInputConnection(vtkReader->GetOutputPort());
  deci->SetTargetReduction(0.6);
  deci->PreserveTopologyOn();

```

- We create a vtkSmoothPolyDataFilter that we set its Input Connection to vtkDecimatePro Outport  
vtkSmartPointer

```auto
  smoother=vtkSmartPointer<vtkSmoothPolyDataFilter>::New();
  smoother->SetInputConnection(deci->GetOutputPort());
  smoother->SetRelaxationFactor(0.8);
  smoother->SetNumberOfIterations(100);
  smoother->BoundarySmoothingOff();
  smoother->FeatureEdgeSmoothingOff();

```

- We create a vtkTransform that we set its Translate to Sizes/2

```auto
  vtkSmartPointer<vtkTransform> transform = vtkSmartPointer<vtkTransform>::New();
  transform->Identity();
  transform->PostMultiply();
  transform->Translate(xSize/2.0,ySize/2.0,zSize/2.0);
  transform->Scale(1,1,1);
  transform->Translate(xSize/2.0,ySize/2.0,zSize/2.0);

```

- We create vtkTransformPolyDataFilter that we set its Input Connection to vtkSmoothPolyDataFilter and its Transform to vtkTransform:

```auto
  vtkSmartPointer<vtkTransformPolyDataFilter> transformFilter = vtkSmartPointer<vtkTransformPolyDataFilter>::New();
  transformFilter->SetInputConnection(smoother->GetOutputPort());
  transformFilter->SetTransform(transform);
  transformFilter->Update();

```

* * *

- Here where we use vtkProbeFilter :

```auto
  vtkSmartPointer<vtkProbeFilter> probe = vtkSmartPointer<vtkProbeFilter>::New();
  probe->SetInputConnection(transformFilter->GetOutputPort());
  probe->SetSourceData(niiReader->GetOutput());
  probe->SpatialMatchOn();
  probe->Update();
  vtkSmartPointer<vtkPolyDataMapper> map = vtkSmartPointer<vtkPolyDataMapper>::New();
  map->SetInputConnection(probe->GetOutputPort());
  map->SetScalarRange(probe->GetOutput()->GetScalarRange());
  vtkSmartPointer<vtkActor> actor = vtkSmartPointer<vtkActor>::New();
  actor->SetMapper(map);

```

---

<div class="post-metadata">

### Author: ![finetjul](https://discourse.vtk.org/user_avatar/discourse.vtk.org/finetjul/32/154_2.png) [@finetjul](https://discourse.vtk.org/u/finetjul)
#### Post date: [November 10, 2022, 12:22pm UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/5 "2022-11-10T12:22:12Z")

</div>

Understood, you have a vtkImageData (.vti or nii) AND a vtkPolyData (.vtp or vtk). That’s what I was missing.

You should be able to do the same in VTK.js.

To replace vtkProbeFilter, you can simply create a point data array (where a scalar is associated at each point), and you manually browse each point (xyz coordinate), convert the coordinate into ijk (`i = (x - origin[0]) / scaling[0]`, `j = (y - origin[1]) / scaling[1]`…) and get the voxel value at this local coordinate.

---

<div class="post-metadata">

### Author: ![peeling](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/p/b5e925/32.png) [@peeling](https://discourse.vtk.org/u/peeling)
#### Post date: [November 14, 2022, 11:01am UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/6 "2022-11-14T11:01:44Z")

</div>

Thank you Julien I’m going to do this

---

<div class="post-metadata">

### Author: ![peeling](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/p/b5e925/32.png) [@peeling](https://discourse.vtk.org/u/peeling)
#### Post date: [November 15, 2022, 3:11pm UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/8 "2022-11-15T15:11:51Z")

</div>

> [@finetjul](#):
>
> get the voxel value at this local coordinate.

Hey Julien,

I have managed browse each point and convert the coordinate, and get the voxel value at this local coordinate,

The next step is directly apply a transformation filter to my vtkPolyData?

---

<div class="post-metadata">

### Author: ![finetjul](https://discourse.vtk.org/user_avatar/discourse.vtk.org/finetjul/32/154_2.png) [@finetjul](https://discourse.vtk.org/u/finetjul)
#### Post date: [November 15, 2022, 3:39pm UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/9 "2022-11-15T15:39:47Z")

</div>

> I have managed browse each point and convert the coordinate, and get the voxel value at this local coordinate,

Great so you have replaced vtkProbeFilter (if you correctly added the voxel value into a `vtkDataArray`( e.g. `vtkDataArray.newInstance({name: 'scalars', values: new UInt16Array(scalarValuesHere), numberOfComponents: 1})`;

> The next step is directly apply a transformation filter to my vtkPolyData?

You can use `vtkMatrixBuilder.buildFromDegree().scale(...).apply(polyData.getPoints().getData())` to apply a transform on ALL the vtkPolyData points.

---

<div class="post-metadata">

### Author: ![peeling](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/p/b5e925/32.png) [@peeling](https://discourse.vtk.org/u/peeling)
#### Post date: [November 16, 2022, 9:46am UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/10 "2022-11-16T09:46:48Z")

</div>

Thank you for your answer,

But `vtkMatrixBuilder.buildFromDegree().scale(...).apply(polyData.getPoints().getData())` the scale function, only takes 3 parameters `(sx,sy,sz)`, where I can use my `vtkDataArray` that has scalars values?

---

<div class="post-metadata">

### Author: ![finetjul](https://discourse.vtk.org/user_avatar/discourse.vtk.org/finetjul/32/154_2.png) [@finetjul](https://discourse.vtk.org/u/finetjul)
#### Post date: [November 16, 2022, 11:15am UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/11 "2022-11-16T11:15:50Z")

</div>

`vtkMatrixBuilder` is for geometrically transforming 3D points (x, y ,z) and not scalar values.  
I do not know what that means to “transform” scalar values.

---

<div class="post-metadata">

### Author: ![peeling](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/p/b5e925/32.png) [@peeling](https://discourse.vtk.org/u/peeling)
#### Post date: [November 16, 2022, 2:30pm UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/12 "2022-11-16T14:30:27Z")

</div>

I’m sorry I didn’t correctly explain my question,  
This is what I did,

```auto

const bounds = reader.VTIgetOutputData().getBounds();
    const scaleParam = 0.5

    const xSize = bounds[1] - bounds[0];
    const ySize = bounds[3] - bounds[2];
    const zSize = bounds[5] - bounds[4]

    const image = readerVTI.getOutputData();
    const polydata = readerVTK.getOutputData();

    vtkMatrixBuilder.buildFromDegree().identity().
    translate(-xSize/2, -ySize/2, -zSize/2).
    scale(scaleParam,scaleParam,scaleParam).
    translate(xSize/2, ySize/2, zSize/2).
    apply(polydata.getPoints().getData());

// Creating a new DataArray that will store voxels (In vtk.js I didn't find setNumberOfTuples() like python and C++, but I continued without)
   const colors = vtkDataArray.newInstance({values: new Uint8Array(polydata.getNumberOfCells()), numberOfComponents:3});

polydata.buildCells();

//Manually browse all the mesh cells, get its coordinates and get the corresponding voxel value in the volume
    for (var currentCellIndex=0; currentCellIndex<polydata.getNumberOfCells(); currentCellIndex++){
      var currentCell = polydata.getCell(currentCellIndex)
      const currentPoints = currentCell.getPoints()
      var currentcellX = 0
      var currentcellY = 0
      var currentcellZ = 0

      const currentCellCoordinate = [
          Math.floor(currentcellX / currentPoints.getNumberOfPoints()),
          Math.floor(currentcellY / currentPoints.getNumberOfPoints()),
          Math.floor(currentcellZ / currentPoints.getNumberOfPoints())
          ]
  
      //'Color cell'
      const currentVxValue = image.getScalarValueFromWorld(currentCellCoordinate[0], currentCellCoordinate[1], currentCellCoordinate[2],0)
// I guess the problem is here because I'm not getting the right voxels values
      colors.setTuple(currentCellIndex, [currentVxValue, currentVxValue, currentVxValue])
    }

    polydata.getCellData().setScalars(colors);
   
    renderer.setBackground([.3, .2, .1]);
    renderer.addActor(actor);
    actor.setMapper(mapper);
    mapper.setInputData(polydata);
    mapper.setColorModeToDirectScalars()
    renderer.resetCamera();
    renderWindow.render();

```

In the end I’m getting like a 1/3 of the head with no colors:  
 ![ezgif-1-1276deb97a](https://discourse.vtk.org/uploads/default/original/2X/e/e079e0daa2009eea5d73e5815e981f2931c5104e.gif)

---

<div class="post-metadata">

### Author: ![finetjul](https://discourse.vtk.org/user_avatar/discourse.vtk.org/finetjul/32/154_2.png) [@finetjul](https://discourse.vtk.org/u/finetjul)
#### Post date: [November 16, 2022, 3:39pm UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/13 "2022-11-16T15:39:09Z")

</div>

I wouldn’t browse cells but only points. I also wouldn’t create an RGB scalar array, but a simple scalar array:

```auto
const points = polyData.getPoints();
const scalars = vtkDataArray.newInstance({dataType: image.getPointData().getDataType(), size: points.getNumberOfPoints(), numberOfComponents:1});

const xyz = [];
for (let i = 0; i < points.getNumberOfPoints(); ++i) {
  points.getPoint(i, xyz);
  const currentVxValue = image.getScalarValueFromWorld(xyz,0);
  scalars.setTuple(i, [currentVxValue]);
}
polydata.getPointData().setScalars(scalars);
...

```

---

<div class="post-metadata">

### Author: ![peeling](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/p/b5e925/32.png) [@peeling](https://discourse.vtk.org/u/peeling)
#### Post date: [November 17, 2022, 9:24am UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/14 "2022-11-17T09:24:52Z")

</div>

Thank you, yes what you wrote is more simple and better, `image.getPointData().getDataType()` is not a function in vtk.js, it doesn’t work, I didn’t use it.  
The result that I’ve obtained is better than the previous, but I don’t get the total of the head

> **[cranky-https-nidv1n - CodeSandbox](https://codesandbox.io/s/cranky-https-nidv1n?file=%2Fsrc%2FApp.js)**
>
> cranky-https-nidv1n by stounej using @emotion/react, @emotion/styled, @kitware/vtk.js, @mui/material, react, react-dom, react-scripts

---

<div class="post-metadata">

### Author: ![finetjul](https://discourse.vtk.org/user_avatar/discourse.vtk.org/finetjul/32/154_2.png) [@finetjul](https://discourse.vtk.org/u/finetjul)
#### Post date: [November 17, 2022, 9:59am UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/15 "2022-11-17T09:59:28Z")

</div>

`pd.getCellData().setScalars(scalars);` → `pd.getPointData().setScalars(scalars);`

---

<div class="post-metadata">

### Author: ![peeling](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/p/b5e925/32.png) [@peeling](https://discourse.vtk.org/u/peeling)
#### Post date: [November 17, 2022, 11:15am UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/16 "2022-11-17T11:15:49Z")

</div>

Yes exactly, I forgot to edit it, thank you so much,  
I don’t know how to solve the problems of Extent (you can see in the CodeSandBox),  
my volume and polydata have not the same Bounds values

```auto
GetScalarPointer: Pixel 113.70,159.73,-5.47 is not in memory. Current extent = 0,181,0,255,0,255

```

I tried to do something like this :  
`image.setExtent(polydata.getBounds())`  
but it seems that is not the right solution.

---

<div class="post-metadata">

### Author: ![finetjul](https://discourse.vtk.org/user_avatar/discourse.vtk.org/finetjul/32/154_2.png) [@finetjul](https://discourse.vtk.org/u/finetjul)
#### Post date: [November 17, 2022, 3:02pm UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/17 "2022-11-17T15:02:10Z")

</div>

You need to invert the matrix multiplication orders:

```auto
vtkMatrixBuilder
              .buildFromDegree()
              .identity()
              .translate(xSize / 2, ySize / 2, zSize / 2)
              .scale(scaleParam, scaleParam, scaleParam)
              .translate(-xSize / 2, -ySize / 2, -zSize / 2)
              .apply(pd.getPoints().getData());

```

---

<div class="post-metadata">

### Author: ![peeling](https://discourse.vtk.org/letter_avatar_proxy/v4/letter/p/b5e925/32.png) [@peeling](https://discourse.vtk.org/u/peeling)
#### Post date: [November 17, 2022, 3:09pm UTC](https://discourse.vtk.org/t/brain-segmentation-on-vtk-head-surface/9706/18 "2022-11-17T15:09:29Z")

</div>

Thank you so much it worked, that was really helpful