@will.schroeder @andreasbuykx : Let me clarify the issue because I feel like it is unclear for now.
What happens is that the value provided as a threshold through setTreshold does not correspond to the value one would get when doing getThresholdedError, but it is more close to threshold*threshold.
Let me even be more precise.
three cases:
- imageA and imageB are the same down to pixels
- imageA and imageB are almost the same, small differences but considered valid
- imageA and imageB are visually different (as in the issue linked)
If you want to know that actual computed image difference between the images, you need to use setThreshold(0) and then use getThresholdedError(). Lets say for our different cases, it will be:
- 0
- 31
- 210
If you want case1 and case2 to be a “sucess” and case 3 to be a “failure”, one would expect that setting threshold to 50 would be a reasonable approach, except it it not !.
Indeed, in the way the threshold is used in the computation, it is more close to be squared, in that case, as if getTresholdedError was 250, which make case 3 a success.
I think this behavior is not documented and very counter intuitive and should be fixed, I took a quick look at the inmplementation but couldnt figure it out in a reasonnnable timeframe
But there is a very simple work around !
Just always use setTreshold(0) , then use if (getThresholdedError() > 50) to determine if the test is a success or a failure.
This is F3D code for checking against baselines (f3d/image.cxx at master · f3d-app/f3d · GitHub):
vtkNew<vtkImageDifference> imDiff;
// handle threshold outside of vtkImageDifference:
// https://gitlab.kitware.com/vtk/vtk/-/issues/18152
imDiff->SetThreshold(0);
imDiff->SetInputConnection(importerThis->GetOutputPort());
imDiff->SetImageConnection(importerRef->GetOutputPort());
imDiff->Update();
error = imDiff->GetThresholdedError();
if (error <= threshold)
{
return true;
}
else
{
return false;
}
I hope it clarifies the issue.