I usually report errors using vtkErrorMacro and capture them in tests using a custom vtkTestingOutputWindow.
The main difference compared to using an explicit error observer (as shown in @lorensen’s example above) is that using a custom output window you capture all errors from all filters and you do not need to add any extra code to your tests (unless you want to check for expected errors) - you just need to add a few extra lines into your CMakeLists.txt.
We have hundreds of tests using this method, see for example how expected errors are defined here.