Overlaying images of organ segmentations with vtkImageBlend

Support
1

Hi,

I’ve got got several vtkImageData containing the segmented organs and would like to overlay them for the planar views in my application.
I’m using a vtkImageBlend and feeding in all the images one after another with the base CT as index 0 so the organs are displayed “in the body”.
I’m setting up a pipeline from the imageBlend to a vtkImageViewer2 for slicing ad displaying the 2d image in my application.

The problem I got right now is that all images of higher index kind of completely override the lower images. I set the blendmode to normal but tried it with compound, too.
Opacity is set to 0.3 for all the images.
The images come from a .nii file.

if _segmentation.scalar == 0:
            self.blendSagittal.SetInputData(_segmentation.imageData)
        else:
            self.blendSagittal.AddInputData(_segmentation.imageData)

        self.blendSagittal.SetOpacity(_segmentation.scalar, _segmentation.opacity)
        self.blendSagittal.Update()
        print(self.blendSagittal)

        if not hasattr(self, 'viewerSagittal'):
            self.viewerSagittal = rendering.renderSagittalViewFromBlend(self.blendSagittal, self.windowSagittal, self.rendererSagittal, 0.5)

This is the section where I add the images to the blend. _segmentation is passed to the function. Scalar functions as a kind of index and the images are passed in order of their scalar. (serves other purposes, too but these aren’t really relevant for this part)
Opacity is set to 0.3 for all of them as mentioned above.

renderSagittalViewFromBlend() just initializes the imageViewer2 and connects the blend to it.

def setupImageBlend():
    imageBlend = vtk.vtkImageBlend()
    imageBlend.SetBlendModeToNormal()
    return imageBlend

This is how I’m currently setting up the imageBlend.

Any suggestions on how to properly overlay all the images?
Also: when I set the opacity of an image to 0 in another process it doesn’t seem to disappear. Is there anything I could to about that?
My thinking was since there is a pipeline between the blend and the viewer it would be enough to set the opacity at the correct index and the update the blend.
Am I missing something?

Thanks for your help!

2

Please don’t comment on all the ways I’m using python wrong. I’ve only been doing python for about 2 months now.

3

The opacities are used as alpha coefficients, they aren’t weighting factors. So setting the opacity=0.33 for three images won’t blend them equally, it’s the ordering of the images in the overlay and the alpha of each overlaid image that is important. I’m about to head out on a trip so that’s the best answer I can give for now, do a search on “alpha compositing” to figure out the math that is involved with the blending.

1 Like
5

Okay, so this puts me one step forward.
What I’m still not understanding:

The dark/black/empty spots in the image (values close to 0 or actually 0) seem to reduce the values of the underlying image when blended. Is there a way to fix that?
I want to properly display one image on top of the other and that doesn’t really work when the empty spots in one subtract from the image below.
At least it looks that way for me.

Another thing:
I’m updating the opacity to 0 which, at least as far as I understand it, should completely “remove” one image from the blend.
According to the log it is set correctly.
I update the blend afterwards.
But in the render window nothing changes.

6

To make black transparent, you will have to set the alpha component to zero in the black areas. I do this with the lookup table, by setting the r,g,b,a=0,0,0,0 for the first entry in the table:

table.Build()
table.SetTableValue(0, 0.0, 0.0, 0.0, 0.0)

Then you can use the table to create an RGBA image to use for blending.

Setting the opacity to 0 for an image should cause the image to be invisible. Are you sure you are setting the opacity for the right image? Can you show your code?

1 Like
7

I’m gonna try using a lookup table for the proper blending.
Haven’t used one before but it shouldn’t be too hard (from a quick look at the docs).

The code for setting the opacity:

    def __setSegmentPlaneVisible(self, _segmentation):
        if _segmentation.visible:
            value = _segmentation.opacity
        else:
            value = 0

        self.planeBlend.SetOpacity(_segmentation.scalar, value)
        self.planeBlend.Update()

        print(self.planeBlend)

The scalar functions as the index of the image in the blend.
And so far, the print output looks correct and the opacity at the correct index changes.

8

I found the solution.
It was really simple and it was probably just my lack of experience with VTK that I didn’t find this earlier:’

Even though there is a pipeline set up from the vtkImageBlend to the vtkImageViewer2 and the blend is updated, this doesn’t meant the imageViewer2 redraws its image.
I had to add the following 3 lines:

        self.viewerSagittal.Render()
        self.viewerFrontal.Render()
        self.viewerTransverse.Render()