VTK Examples - Testing for the C++ examples now uses SSIM

Announcements
1

Introduction

The VTK C++ Examples are now using the Structural Similarity Index (SSIM) which is a metric for measuring the perceived quality and similarity of two images.

Some images needed to be replaced due to the more stringent SSIM testing, mostly due to changes in color.

In summary, 98% or more tests pass in both Linux and Windows. The tests were done on the same machine using Debug builds for VTK and the VTK Examples. For Linux, two tests failed out of 1023 and for Windows using Visual Studio 2026, sixteen tests failed out of 1021. For windows ten failures generated this message on exit:

Expression: cannot compare incompatible string_view iterators for equality.

An issue for this has been raised, please see: Bug in vtkBYUReader and vtkSLCReader.

If the image comparison fails, three files are generated, two PNG files consisting of the current image and a difference image, along with a VTI file. The VTI file has has three slices in the XY plane that can be viewed using ParaView called:

  • Baseline, colored by Baseline Magnitude.
  • Current, colored by Current Magnitude.
  • ImageScalars, colored by Image Scalars Magnitude.

Can anyone provide more information about this file?

Also if you are interested, please see Do we need a testing framework example?.

Linux

99% tests passed, 2 tests failed out of 1023.

  • IndividualVRML now cannot run teapot.wrl as a test, it crashes for some reason, so I created utah_teapot.wrl from a recent version of utah_teapot.obj for testing purposes.

All tests pass except:

  • Images-ImageContinuousDilate3D (Subprocess aborted)
  • Images-ImageContinuousDilate3D (Subprocess aborted)

The crash happens at line 79 in vtkDataArrayValueRange_AOS.h when the PNG file is used.

Windows

98% tests passed, 16 tests failed out of 1021.

Nine failures are due to the usage of the vtkBYUReader and one due to the use of the vtkSLCReader please see: Bug in vtkBYUReader and vtkSLCReader

As in Linux, these two tests also fail:

  • Images-ImageContinuousDilate3D (Subprocess aborted)
  • Images-ImageContinuousDilate3D (Subprocess aborted)

The crash happens at line 79 in vtkDataArrayValueRange_AOS.h when the PNG file is used.

Points-SignedDistance needs VTK_TESTING_IMAGE_COMPARE_METHOD=LOOSE_VALID in order to pass.

InfoVis-XGMLReader fails in windows even when VTK_TESTING_IMAGE_COMPARE_METHOD=LEGACY_VALID is set. TIGHT_VALID is Ok in Linux.

2

Sounds like a bug to fix.

3

Please see IndividualVRML failure