Brush tool only paints correctly in one view (VTKPolyDataToImageStencil issue?)

Support
1

I’m implementing a brush tool for my VTK-based application, which has three 2D views: axial, sagittal, and coronal.

The brush works as follows: while the left mouse button is held and the mouse moves, I create a circular brush and add it to a vtkAppendPolyData. The relevant code looks like this:

auto transform = vtkSmartPointer<vtkTransform>::New();
transform->Translate(worldCoords[0], worldCoords[1], worldCoords[2]);

if (context->getDisplayType() == DisplayType::DISPLAY_2D) {
    VTKDisplay2D* display2D = dynamic_cast<VTKDisplay2D*>(context);
    switch (display2D->getCurrentAxis()) {
    case CORONAL:
        transform->RotateX(90.0);
        break;
    case SAGITTAL:
        transform->RotateY(90.0);
        break;
    case AXIAL:
        break;
    }
}

UpdateBrushPosition(context);

auto transformFilter = vtkSmartPointer<vtkTransformPolyDataFilter>::New();
transformFilter->SetInputConnection(brushPolyData->GetOutputPort());
transformFilter->SetTransform(transform);
transformFilter->Update();

appendPolyData->AddInputData(transformFilter->GetOutput());

This part works fine — when I print the final polygon bounds, they make sense and the geometry looks correct.

When I release the left mouse button, I use the accumulated polygons to paint the corresponding voxels in a segmentation image:

if (appendPolyData->GetNumberOfInputConnections(0) > 0) {

    appendPolyData->Update();
    polyDataToImageStencil->Update();
            
    vtkImageStencilData* stencilData = polyDataToImageStencil->GetOutput();
    
    int* extent = stencilData->GetExtent();

    for (int z = extent[4]; z <= extent[5]; z++) {
        for (int y = extent[2]; y <= extent[3]; y++) {
            int x1, x2;
			for (int iter = 0; stencilData->GetNextExtent(x1, x2, extent[0], extent[1], y, z, iter); iter++) {
                for (int x = x1; x <= x2; x++) {
                    segmentationImage->SetScalarComponentFromDouble(x, y, z, 0, 1);
                }
            }
        }
    }
    
    segmentationImage->Modified();
    context->getWorkspaceContext()->getActiveSegmentation()->setSegmentation(segmentationImage);
    
    context->getInteractor()->GetRenderWindow()->Render();
}

appendPolyData->RemoveAllInputs();

The problem:
The painting only works in the sagittal view. In that view, the voxels are updated correctly and the other two views also display the updated segmentation (since they share the same vtkImageData).

However, when I try painting from the axial or coronal views, the inner loop (for (int x = x1; x <= x2; x++)) never executes — meaning that GetNextExtent() returns no valid extents. You can see the behavior in this video:

I’ve read in the vtkPolyDataToImageStencil documentation that:

Warning:
If contours are provided, they must be aligned with the Z planes. Other contour orientations are not supported.

I’m wondering if this might be the issue — but it seems odd, because the axial view (which corresponds to constant Z) is actually the one where it doesn’t work.

Has anyone encountered this issue or can confirm whether the vtkPolyDataToImageStencil really requires the input polygons to be aligned with the Z plane?
Any advice or workaround would be greatly appreciated.

2

I strongly recommend vtkROIStencilSource for this, instead of vtkPolyDataToImageStencil. The vtkROIStencilSource can produce cylinders in the axial, sagittal, or coronal orientations and a cylinder is actually what you want… a cylinder that is one slice thick (i.e. a thin disk).

The Append operation is why you see strange behavior with vtkPolyDataToImageStencil. The polygons that you input to vtkPolyDataToImageStencil are supposed to form a closed surface. When you append polygons in the sagittal direction, vtkPolyDataToImageStencil will “stencil” the voxels between adjacent polygons that have been appended. Essentially, instead of stencilling each polygon individually (which it cannot do), it stencils the space between the polygons.

For the axial and coronal directions, appending the polygons just creates a mess of intersecting polygons that vtkPolyDataToImageStencil cannot deal with. That’s why it isn’t working in those directions.

In my opinion, vtkAppendPolyData should not be part of your pipeline. You want to combine the brush positions in your image space, instead.

To efficiently “draw” the stencil into an image, you can use vtkImageStencilIterator. This iterator class is used, for example, in the vtkImageStencilToImage class. It is used roughly as follows:

  vtkImageStencilIterator<unsigned char> outIter(imageData, stencilData);

  // Loop through image voxels and apply the stencil
  while (!outIter.IsAtEnd())
  {
    unsigned char* outPtr = outIter.BeginSpan();
    unsigned char* spanEndPtr = outIter.EndSpan();

    if (outIter.IsInStencil()) {
      while (outPtr != spanEndPtr) {
        *outPtr++ = 1; // apply the brush color
      }
    }

    outIter.NextSpan();
  }

In summary, my advice is to use vtkROIStencilSource and vtkImageStencilIterator. Don’t use vtkPolyDataToImageStencil or vtkAppendFilter.

3

I finally got this working, thanks to your help!

At first, I tried to create a single polygon from all the brush circles, but I didn’t account for cases where the circles don’t intersect — that ended up creating an incorrect polygon that filled the gaps between them.

In the end, I implemented it like this:

if (!currentStroke.empty() && segmentationImage) {
    for (const auto& stroke : currentStroke) {
        if (context->getDisplayType() == DisplayType::DISPLAY_2D) {
            VTKDisplay2D* display2D = dynamic_cast<VTKDisplay2D*>(context);
            
            switch (display2D->getCurrentAxis()) {
            case AXIAL:
                roiStencilSource->SetShapeToCylinderZ();
                roiStencilSource->SetBounds(
                    stroke.worldX - brushRadius, stroke.worldX + brushRadius,
                    stroke.worldY - brushRadius, stroke.worldY + brushRadius,
                    segmentationImage->GetOrigin()[2] + stroke.slice * segmentationImage->GetSpacing()[2],
                    segmentationImage->GetOrigin()[2] + stroke.slice * segmentationImage->GetSpacing()[2]);
                break;
            case CORONAL:
                roiStencilSource->SetShapeToCylinderY();
                roiStencilSource->SetBounds(
                    stroke.worldX - brushRadius, stroke.worldX + brushRadius,
                    segmentationImage->GetOrigin()[1] + stroke.slice * segmentationImage->GetSpacing()[1],
                    segmentationImage->GetOrigin()[1] + stroke.slice * segmentationImage->GetSpacing()[1],
                    stroke.worldZ - brushRadius, stroke.worldZ + brushRadius);
                break;
            case SAGITTAL:
                roiStencilSource->SetShapeToCylinderX();
                roiStencilSource->SetBounds(
                    segmentationImage->GetOrigin()[0] + stroke.slice * segmentationImage->GetSpacing()[0],
                    segmentationImage->GetOrigin()[0] + stroke.slice * segmentationImage->GetSpacing()[0],
                    stroke.worldY - brushRadius, stroke.worldY + brushRadius,
                    stroke.worldZ - brushRadius, stroke.worldZ + brushRadius);
                break;
            }

            roiStencilSource->Update();
            vtkImageStencilData* stencilData = roiStencilSource->GetOutput();


            //https://discourse.vtk.org/t/brush-tool-only-paints-correctly-in-one-view-vtkpolydatatoimagestencil-issue/16114
            vtkImageStencilIterator<int> outIter(segmentationImage, stencilData);

            while (!outIter.IsAtEnd()) {
                int* outPtr = outIter.BeginSpan();
                int* spanEndPtr = outIter.EndSpan();

                if (outIter.IsInStencil()) {
                    while (outPtr != spanEndPtr) {
                        *outPtr++ = 1;
                    }
                }
                outIter.NextSpan();
            }
        }
    }

Basically, I just store the circle positions and their corresponding slices, and then on LeftButtonUp I iterate through them and draw each one using vtkROIStencilSource.

It’s working fine, but I’m wondering if there’s a cleaner or more efficient way to do this — maybe a more “VTK-native” approach to combine these strokes into a single stencil?

4

To give me a better idea, can you tell me which of this is correct?

  1. The currentStroke goes from the previous mouse position to the current mouse position
  2. The currentStroke includes all mouse positions from when the mouse button was pressed until the mouse button was released

The reason I ask, is that for (2) things might get very slow for long strokes, assuming that the entire currentStroke is updated every time the mouse position changes. Basically, you want to minimize the amount of redrawing that occurs per mouse move. The segmentationImage keeps track of what voxels have already been “hit” by the brush.

To answer your question, the vtkImageStencilData::Add() method can be used to update one stencil by merging in a different stencil. This would be used as follows:

  1. First, create an empty vtkImageStencilData that has the same Extent as your entire image
  2. Every time the brush position changes, update the vtkROIStencilSource and add its output to the stencil created in (1).
5

The currentStroke is a std::vector that includes all mouse positions, but I only draw them onLeftButtonUp, but I guess there can be an overlapping if two ROI are intersected and then we do redrawing on these intersecting voxels. segmentationImage is actually the volume that I am trying to paint to.

6

Ah, yes, it makes sense if you only draw on ButtonUp. I was thinking that you might be drawing each time the mouse changes position.

The redrawing of previously-drawn voxels shouldn’t slow things down much, but the vtkImageStencilData::Add() method that I mentioned could be used to avoid redrawing the voxels (though I can’t be sure whether it would actually speed things up).

7

Okay I guess im going to leave it like it is.
Thank you so much David!