vtkImageDifference does not detect different images

Development
1

Hi all,

In the CI of Themys, our ParaView Branded Application, we use the module Testing.py which makes use of the vtkImageDifference algorithm.

It seems that with default configuration this algorithm does not detect that the two images below are different.

hic_different_reloads_0
hic_different_reloads_01356×693 20 KB

hic_different_reloads_0.valid
hic_different_reloads_0.valid1356×693 24.4 KB

After reading vtkImageDifference does not detect image differences - #6 by mwestphal i tried to add idiff.SetThreshold(0) in Testing.py and with this modification the images are correctfully detected as different.

I do not know if it is a known corner case of the vtkImageDifference algorithm or a bug.
By the way should i use a custom python script with idiff.SetThreshold?

Thanks in advance

G.

2

It is a known issue. We are currently in the process of changing the metric used for image comparison in regression testing (see here). This will be quite impactful for VTK and projects depending on it so it will be opt in at first. The current metric is too forgiving and does not capture enough information in the image to be satisfactory.

3

Ok. Thanks for this quick answer @Yohann_Bearzi

4

TBH making the treshold default to zero and improving the doc would be a very simple test, especially considering that the new class will not replace the old class for compat reasons.

5

@mwestphal @Yohann_Bearzi If i understand and read correctly the source code, i can have the strictest options for comparison by typing:

    # Here we want to be as strict as possible so do not set threshold
    idiff.SetThreshold(0)
    # Do not compare images by allowing a shift of two pixels between images
    idiff.SetAllowShift(False)
    # Do not compare averaged 3x3 data between the images
    idiff.SetAveraging(False)

Am i right?

6

Indeed, but expect the result to almost never be zero.

7

Ok. Thanks @mwestphal