I am developing a python-VTK application and am noticing some unusual behavior in python unittest-based tests I am building alongside my application. I have noticed that tests that are defined in separate suites appear to interfere with each other and my theory is that the underlying python VTK functionality is using some global or otherwise shared information making the scoping of my python tests not as isolated as I’d like. Ultimately I’m looking for tips/guidance on how to correctly set up python unit tests that leverage python VTK in a safe and isolated manner.
I have a couple anecdotal examples, first I noticed that unit tests which pass when run once will sometimes fail if running them multiple times per python interpreter. I have found that after a dozen or so executions of the same test, I’d receive a runtime failure that looks like this (python VTK 9.5.1 on Mac):
Through much testing I discovered that this error can be avoided 100% of the time by turning off python garbage collection with import gc; gc.disable() at the top of my testing script. The effectiveness of this approach has led me to believe that there is something unsafe happening when the python script creates multiple full rendering pipelines and those python objects call underlying C++ libraries to deconstruct. Telling python to never clean up bypasses the issue.
Another example - I have two suites, one of which uses self.render_window.OffScreenRenderingOn() and the other does not. If both tests are run in the same python instance, I notice the offscreen rendering test actually renders to the screen when it shouldn’t. Only when both tests are in the same file will this occur.
I can work on getting a standalone example of this to share but in the meantime I’m curious if anyone out there has run into this or has any tips on isolation and memory safety when running python VTK based unit tests. Thanks!
For the runtime failures, do the stack traces always show InvokeEvent? I suspect that events are being invoked after the observer has destructed. This would explain why disabling gc helps. If you can confirm that InvokeEvent is always involved, I can dig deeper to find the cause.
Regarding the offscreen rendering, I’m not very familiar with that part of the VTK code, but I can imagine that it might involve some global state. VTK’s own test suite always runs each test in its own process.
When I first ran into this issue I ran it by a couple AIs which advised me to better manage tearing down VTK-managed objects myself. This led me to create a tear_down method that attempts to deconstruct everything I use in my top level python class, it looks like this:
def tear_down(self):
if self.controller.timer_id:
self.interactor.DestroyTimer(self.controller.timer_id)
for camera in self.controller.cameras:
self.controller.cameras[camera].RemoveAllObservers()
self.interactor.RemoveAllObservers()
for renderer in self.renderers:
self.renderers[renderer].RemoveAllObservers()
# Mark for deletion
del self.interactor
self.interactor = None
for renderer in self.renderers:
del renderer
Calling the function however appeared to make the problem worse and would result in another type of error signature (in addition to the first one) that looks like this:
Backing this out seemed to completely remove this error signature - from this I concluded that 1. I clearly don’t know the safe way to deconstruct VTK objects in python and 2. AI will not be taking over the world with software anytime soon
VTK’s own test suite always runs each test in its own process.
Thanks for this info - I could do as well on the python side if I absolutely had to.
