Hi all,
I’m displaying 2D images using QVTKNativeWidget and I’m seeing zigzag/jagged edges along high-contrast boundaries. I’ve attached two screenshots:
-
Current issue: edges look stair-stepped/pixelated
-
Desired output: smooth edges
Two things are needed to achieve smooth edges:
- high-order interpolation (e.g. cubic, b-spline, or sinc interpolation)
- floating-point data (to avoid discretization following the interpolation)
This can be achieved with vtkImageResliceMapper if the right settings are used.
Can you briefly describe the pipeline that you’re currently using to render the images?
1 Like
thanks for the reply, I tried your suggestion but couldn’t achieve the required smoothness. In another project, they accomplished this by editing vtkOpenGLImageSliceMapper in the VTK source(VTK v5). Is there a way to achieve similar smoothness using vtkOpenGLImageSliceMapper (in VTK9)?
There is no easy way achieve what you want with vtkImageSlicerMapper or with vtkOpenGLImageSliceMapper. That’s why I wrote the vtkImageResliceMapper, because it can do things that vtkImageSliceMapper cannot do. This time I have included a reasonably sophisticated example below (click the arrow to expand).
ResliceMapper2D python example
#! /usr/bin/env python
"""
This is a program to demonstrate 2D image display with the
vtkImageResliceMapper class. If given no filename, it defaults
to generating a simple image of a disk. If given a filename,
the file must be a 16-bit png, assumed to be a CT image.
"""
from vtkmodules.vtkCommonCore import vtkLookupTable
from vtkmodules.vtkCommonDataModel import vtkSphere
from vtkmodules.vtkImagingHybrid import vtkSampleFunction
from vtkmodules.vtkImagingMath import vtkImageMathematics
from vtkmodules.vtkIOImage import vtkPNGReader
from vtkmodules.vtkInteractionStyle import vtkInteractorStyleImage
from vtkmodules.vtkRenderingImage import vtkImageResliceMapper
import vtkmodules.vtkInteractionStyle
import vtkmodules.vtkRenderingFreeType
import vtkmodules.vtkRenderingOpenGL2
from vtkmodules.vtkImagingCore import (
vtkImageBSplineCoefficients,
vtkImageBSplineInterpolator,
vtkImageSincInterpolator,
vtkImageShiftScale,
vtkImageThreshold,
)
from vtkmodules.vtkRenderingCore import (
vtkImageProperty,
vtkImageSlice,
vtkRenderWindow,
vtkRenderWindowInteractor,
vtkRenderer,
)
import os
import sys
import math
import argparse
def makeDiskImage(dims=(256, 256), r=10.0):
"""Generate an image of a disk, type 'short', range [-1000,3000]."""
# we want the disk to have a smooth boundary, so we make an
# image where the pixel value is the signed distance from the
# edge of the disk, and then threshold the pixel values
# between -1 and +1 to get a linear ramp at the edge
# use vtkSphere to create a map of distance squared from center
sphere = vtkSphere()
sphere.SetRadius(0.0)
sphere.SetCenter((dims[0] - 1)/2.0, (dims[1] - 1)/2.0, 0.0)
# sample vtkSphere to get an image of distance squared
sample = vtkSampleFunction()
sample.SetImplicitFunction(sphere)
sample.SetOutputScalarTypeToFloat()
sample.SetSampleDimensions(dims[0], dims[1], 1)
sample.SetModelBounds(0, dims[0]-1, 0, dims[1]-1, 0, 0)
sample.ComputeNormalsOff()
# take square root to get distance from center
radius = vtkImageMathematics()
radius.SetInputConnection(sample.GetOutputPort())
radius.SetOperationToSquareRoot()
# subtract radius to get distance from edge of disk
subtract = vtkImageMathematics()
subtract.SetInputConnection(radius.GetOutputPort())
subtract.SetOperationToAddConstant()
subtract.SetConstantC(-r)
# clamp distance map (upper clamp)
thresh1 = vtkImageThreshold()
thresh1.SetInputConnection(subtract.GetOutputPort())
thresh1.ThresholdByUpper(1.0)
thresh1.SetInValue(1.0)
thresh1.ReplaceInOn()
# clamp distance map (lower clamp)
thresh2 = vtkImageThreshold()
thresh2.SetInputConnection(thresh1.GetOutputPort())
thresh2.ThresholdByLower(-1.0)
thresh2.SetInValue(-1.0)
thresh2.ReplaceInOn()
# create a 16-bit image with range [0, 2000], note that we
# invert the image so that 'inside' is bright
convert = vtkImageShiftScale()
convert.SetInputConnection(thresh2.GetOutputPort())
convert.SetShift(-0.5)
convert.SetScale(-2000.0)
convert.SetOutputScalarTypeToShort()
# return the image
convert.Update()
return convert.GetOutput()
def main(quality='high',
interpolation='linear',
focus=None,
zoom=10.0,
fullscale=False,
filename=None):
if filename is not None:
reader = vtkPNGReader()
if not reader.CanReadFile(filename):
print(f"Cannot read file \"{filename}\".")
return
reader.SetFileName(filename)
reader.Update()
imageToInterpolate = reader.GetOutput()
else:
# generate a high-contrast test image
imageToInterpolate = makeDiskImage()
dataRange = imageToInterpolate.GetPointData().GetScalars().GetRange()
print('range', dataRange)
if dataRange[1] - dataRange[0] < 256:
print("Warning: this is not a 16-bit image!\n")
size = imageToInterpolate.GetDimensions()[0:2]
print('size', size)
# create the interactor
style = vtkInteractorStyleImage()
style.SetInteractionModeToImage2D()
iren = vtkRenderWindowInteractor()
iren.SetInteractorStyle(style)
# create the renderer and window
renderer = vtkRenderer()
rw = vtkRenderWindow()
rw.SetSize(size)
rw.AddRenderer(renderer)
rw.SetInteractor(iren)
# make a grayscale lookup table
table = vtkLookupTable()
table.SetValueRange(0.0, 1.0)
table.SetSaturationRange(0.0, 0.0)
table.SetHueRange(0.0, 0.0)
table.SetAlphaRange(0.0, 1.0)
table.SetRampToLinear()
table.Build()
# make the mapper
im = vtkImageResliceMapper()
im.SetInputData(imageToInterpolate)
im.SliceFacesCameraOn()
im.SliceAtFocalPointOn()
im.JumpToNearestSliceOn()
im.SeparateWindowLevelOperationOn()
if quality == 'auto':
im.AutoAdjustImageQualityOn()
im.ResampleToScreenPixelsOn()
elif quality == 'high':
im.AutoAdjustImageQualityOff()
im.ResampleToScreenPixelsOn()
elif quality == 'low':
im.AutoAdjustImageQualityOff()
im.ResampleToScreenPixelsOff()
# make the property
ip = vtkImageProperty()
ip.SetLookupTable(table)
if fullscale:
# full range
ip.SetColorWindow(dataRange[1] - dataRange[0])
ip.SetColorLevel(0.5*(dataRange[0] + dataRange[1]))
elif dataRange[0] < -100.0:
# tight window (signed data)
ip.SetColorWindow(255.0)
ip.SetColorLevel(-0.5)
else:
# tight window (unsigned data)
ip.SetColorWindow(255.0)
ip.SetColorLevel(999.5)
# make the actor
ia = vtkImageSlice()
ia.SetMapper(im)
ia.SetProperty(ip)
renderer.AddViewProp(ia)
# default is linear interpolation
if interpolation == 'linear':
ip.SetInterpolationTypeToLinear()
# nearest-neighbor interpolation
if interpolation == 'nearest':
ip.SetInterpolationTypeToNearest()
# cubic interpolation
if interpolation == 'cubic':
ip.SetInterpolationTypeToCubic()
# b-spline interpolation
if interpolation == 'bspline':
bspline = vtkImageBSplineCoefficients()
bspline.SetSplineDegree(3)
bspline.SetInputData(imageToInterpolate)
bspline.Update();
interp = vtkImageBSplineInterpolator()
interp.SetSplineDegree(3)
im.SetInterpolator(interp)
im.SetInputData(bspline.GetOutput())
# sinc interpolation
if interpolation == 'sinc':
interp = vtkImageSincInterpolator()
interp.SetWindowFunctionToBlackman()
im.SetInterpolator(interp)
# sinc interpolation with antialiasing
# (takes effect when image pixels are smaller than screen pixels,
# for example when the zoom factor is smaller than 1.0)
if interpolation == 'antialiased':
interp = vtkImageSincInterpolator()
interp.SetWindowFunctionToBlackman()
interp.AntialiasingOn()
im.SetInterpolator(interp)
# default focus to center of the image
bounds = imageToInterpolate.GetBounds()
if not focus:
focusX = 0.5*(bounds[0] + bounds[1])
focusY = 0.5*(bounds[2] + bounds[3])
focusZ = 0.5*(bounds[4] + bounds[5])
else:
focusX, focusY = focus
focusZ = 0.5*(bounds[4] + bounds[5])
# camera setup
camera = renderer.GetActiveCamera()
camera.ParallelProjectionOn()
camera.SetParallelScale(0.5*(bounds[3] - bounds[2])/zoom)
camera.SetFocalPoint(focusX, focusY, focusZ)
camera.SetPosition(focusX, focusY, focusZ + 1.0)
camera.SetClippingRange(0.5, 1.5)
# render the image, start the interactor
rw.Render()
iren.Start()
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="ResliceMapper example.")
parser.add_argument('--quality', required=False,
choices=["auto", "high", "low"],
help="Rendering quality.")
parser.add_argument('--interpolation', required=False,
choices=["nearest", "linear", "cubic", "bspline",
"sinc", "antialiased"],
help="Interpolator to use.")
parser.add_argument("--fullscale", required=False, action="count",
help="View full data range (do not window).")
parser.add_argument("filename", nargs="*",
help="PNG image (must be 16-bit)")
args = parser.parse_args()
if args.filename:
filename = args.filename[0]
if not os.path.isfile(filename):
sys.stderr.write(f"Cannot find file: \"{filename}\"\n")
sys.exit(1)
if os.path.splitext(filename)[1] not in (".png", ".PNG"):
sys.stderr.write(f"Expected a png: \"{filename}\"\n")
sys.exit(1)
else:
filename = None
main(quality=args.quality,
interpolation=args.interpolation,
fullscale=args.fullscale,
filename=filename)
The trick to achieving high-quality image display is to have an image rendering pipeline that applies the steps in the right order:
- interpolation (zoom and optionally slicing for 3D images)
- window/level (with optional color table)
The problem with vtkImageSliceMapper is that it applies the operations in the wrong order. First it does window/level, and then it lets the GPU perform linear interpolation. In comparison, vtkImageResliceMapper applies the operations in the correct order (as long as ResampleToScreenPixels is On, which is the default for this mapper). Other benefits of vtkImageResliceMapper is that it uses floating-point for all intermediate calculations, and it allows higher-order interpolation.
Here are some sample images to demonstrate the difference. First, here’s what the result of a window/level operations looks like (without any interpolation):
Next, here’s the difference between interpolating after window/level versus interpolating before window/level. The image on the left is what you get with vtkImageSliceMapper, the images on the right are what you get with vtkImageResliceMapper.
As you can see, the jagged edges are the result of doing the operations in the wrong order. Essentially, the window/level reduces the dynamic range of the data and this mucks up the interpolation. It’s the same basic problem as distortion due to clipping for audio signals: any digital signal needs to have sufficient headroom and footroom or else even basic filtering operations like interpolation can result in distortion.
Ideally, someone could contribute a fixed vtkOpenGLImageSliceMapper back to VTK. The fix would be roughly as follows: load the full dynamic range image onto the GPU (either as short, unsigned short, or float), and provide shader programs for interpolation and window/level (plus colormap) on the full dynamic range data.
2 Likes