Simplification of Python modules, and things to watch for

Hi All,

This morning I merged a branch into master that changes the way that VTK’s Python modules are built and loaded.

As you might recall, the vtk-python modules used to have three components:

  • libvtkCommonCorePythonD.so
  • vtkCommonCorePython.so
  • vtkCommonCore.py

The “.py” module loaded the “Python.so” module, which was dynamically linked to the PythonD.so library, which is where the PyTypeObjects for the vtk classes were defined.

For VTK 8.90, The “PythonD.so” libraries were absorbed by “Python.so”. This took some effort, because the PyTypeObject structs have to be able to resolve symbols between each other. All the dependent shared object loading and symbol resolution that the dynamic linker had previously done automatically, now had to be done explicitly. But the result was fewer files:

  • vtkCommonCorePython.so
  • vtkCommonCore.py

Finally, as of this morning, the “Python” suffix has been removed from the modules. This means that the “.py” files are no longer needed:

  • vtkCommonCore.so

Now if you do “import vtkmodules.vtkCommonCore”, you are directly loading the module.

Note that static builds of the wrappers are still handled as they were before. If you do a static build, then the wrapper code is built as “libvtkCommonCorePython.a”, and a “vtkCommonCore.py” stub is created.


Here are the things to watch out for. If you update your source tree from a previous version of VTK to this new master (and, of course, if you build the Python wrappers), then you must do a “make clean” or equivalent. If you don’t, then your build tree will contain both the old “vtkComonCorePython.so” and the new “vtkCommonCore.so”.

A side effect of this is that the dashboard is probably going to show spurious Python-related test failures for a while, since switching between builds that contain this change and builds that don’t will result in dirty build directories. If you see such test failures in your own MR, then rebase your MR on master and tell the buildbots to do a clean build (test --clean).

4 Likes

very neat! Thanks for this cleanup, @dgobbi!