Issue with using vtkPythonUtil::GetObjectFromPointer for custom VTK classes

jcfr · August 29, 2020, 2:48pm

Working on updating Slicer to build against the latest VTK (see [1]),
I noticed that VTK object returned by Qt slots only include vtkObject methods.

Question: While I am investigating further, does anyone have ideas on what may be wrong ?

To help with this, you will find below some more context in the form of a FAQ.

[1] https://github.com/Slicer/Slicer/pull/5141

FAQ

What is the problem ?

With Slicer built against VTK 9.0.20200825, custom VTK C++ class instances wrapped using vtkPythonUtil::GetObjectFromPointer do not include custom methods:

>>> slicer.app.mrmlScene
<qt slot mrmlScene of qSlicerApplication instance at 0x7fc242b98e88>

>>> slicer.app.mrmlScene()
(vtkmodules.vtkCommonCore.vtkObject)0x7fc22be10f48

>>> 'GetNodes' in dir(slicer.app.mrmlScene())
False

For comparison, here is what we have with Slicer built against VTK 8.2:

>>> slicer.app.mrmlScene
<qt slot mrmlScene of qSlicerApplication instance at 0x7f9d952fbf48>

>>> slicer.app.mrmlScene()
(MRMLCorePython.vtkMRMLScene)0x7f9d8213f768

>>> 'GetNodes' in dir(slicer.app.mrmlScene())
True

What make you think the custom classes are properly wrapped ?

Importing the object from python interpreter allow to confirm the object specific methods are available.

from python interpreter:

$ ../python-install/bin/PythonSlicer
>>> import mrml
>>> 'GetNodes' in dir(mrml.vtkMRMLScene)
True

from embedded python interpreter

from application using executeString

>>> slicer.app.pythonManager().executeString("import mrml; print('GetNodes' in dir(mrml.vtkMRMLScene))")
True

Context: Slicer built against VTK 9.0.20200825

Any idea on where the problem could be ?

Since the custom class is properly wrapped when imported from a python interpreter, I suspect the expected base class is not found in ClassMap

github.com

Slicer/VTK/blob/8ec81d403c4d87a2787fc3e03e2087d812d05d0b/Wrapping/PythonCore/vtkPythonUtil.cxx#L407-L438


PyObject* vtkPythonUtil::GetObjectFromPointer(vtkObjectBase* ptr)
{
  PyObject* obj = vtkPythonUtil::FindObject(ptr);

  if (obj == nullptr)
  {
    // create a new object
    PyVTKClass* vtkclass = nullptr;
    vtkPythonClassMap::iterator k = vtkPythonMap->ClassMap->find(ptr->GetClassName());
    if (k != vtkPythonMap->ClassMap->end())
    {
      vtkclass = &k->second;
    }

    // if the class was not in the map, then find the nearest base class
    // that is, and associate ptr->GetClassName() with that base class
    if (vtkclass == nullptr)
    {
      const char* classname = ptr->GetClassName();
      vtkclass = vtkPythonUtil::FindNearestBaseClass(ptr);

This file has been truncated. show original

Context: Slicer built against VTK 9.0.20200825

How are the custom VTK classes wrapped ?

We wrap our custom VTK classes using vtkMacroKitPythonWrap.cmake as well as modified version of vtkWrapPython.cmake and vtkWrapHierarchy.cmake adapted from prior version of VTK.

Context: Slicer built against VTK 9.0.20200825

How are PythonQt and VTK integrated ?

Few years ago, we worked with the maintainer of PythonQt to add the concept of PythonQtForeignWrapperFactory. To ensure VTK object associated with Qt signals, slots and properties are properly wrapped/unwrapped, we then implemented [ctkVTKPythonQtWrapperFactory](https://github.com/commontk/CTK/blob/support-build-with-vtk89/Libs/Visualization/VTK/Core/ctkVTKPythonQtWrapperFactory.cpp) distributed in CTK`.

Context: Slicer built against VTK 9.0.20200825

cc: @dgobbi @lassoan @pieper

dgobbi · August 29, 2020, 5:03pm

Given that GetObjectFromPointer() relies on GetClassName(), what does this return?

slicer.app.mrmlScene().GetClassName()

If that returns the correct class name, then yes, the ClassMap is the most likely culprit. The worst possibility is that there are multiple instances of the ClassMap in Slicer. The singleton that contains the ClassMap is created in the vtkWrappingPythonCore library:

// constructs the singleton
void vtkPythonUtilCreateIfNeeded()
{
  if (vtkPythonMap == nullptr)
  {
    vtkPythonMap = new vtkPythonUtil();
    Py_AtExit(vtkPythonUtilDelete);
  }
}

The dynamic library handling for the wrappers changed in VTK 9 as compared to VTK 8. There used to be “vtkCommonCorePythonD” library in addition to the “vtkCommonCorePython” module.

The old “PythonD” libraries relied on the system loader to ensure that dependent libraries were loaded, e.g. to ensure that vtkCommonCorePythonD would be loaded if vtkCommonMiscPythonD was loaded. This is important because these “D” libraries contained the actual Python extension classes for VTK. So if you imported vtkCommonMisc, then the wrappers would automatically know about all the classes in vtkCommonCore even if you didn’t import vtkCommonCore.

In VTK 9, the dependency handling has changed. In VTK 9, when you import vtkCommonMisc, the initCommonMisc() method will import vtkmodules.vtkCommonCore and all other dependencies. So dependencies are handled through pythonic means, rather than being handled by the system loader.

I’m not saying that this change to the module loading is necessarily related to your custom class issue, but it is something to keep in mind while you are debugging. Take a look in the InitImpl.cxx code for your custom Python modules to make sure that dependencies are properly loaded.

It should go without saying that if you call slicer.app.mrmlScene() without first importing your mrml wrappers into Python, then a vtkObject is all you should expect to get.

Another possibility, though unlikely, is that multiple inheritance is gumming things up. Do your custom-wrapped objects use multiple inheritance?

jcfr · August 29, 2020, 5:14pm

Update: The wrapping infrastructure does not detect that our custom classes should be wrapped as VTK classes.

Details below

It turns out that with VTK 8.2.0, the custom class wrappers are properly generated and ends up calling PyVTKClass_Add that is responsible to update the vtkPythonMap singleton by calling vtkPythonUtil::AddClassToMap:

// Libs/MRML/Core/vtkMRMLScenePython.cxx`

[...]

PyObject *PyvtkMRMLScene_ClassNew()
{
  PyVTKClass_Add(
    &PyvtkMRMLScene_Type, PyvtkMRMLScene_Methods,
    "vtkMRMLScene",
 &PyvtkMRMLScene_StaticNew);
 [...]

void PyVTKAddFile_vtkMRMLScene(PyObject *dict)
{
  PyObject *o;
  o = PyvtkMRMLScene_ClassNew();

  if (o && PyDict_SetItemString(dict, "vtkMRMLScene", o) != 0)
  {
    Py_DECREF(o);
  }
}

[...]

whereas when building against VTK 9.0.20200825, the following wrappers are generated:

// Libs/MRML/Core/vtkMRMLScenePython.cxx

[...]

void PyVTKAddFile_vtkMRMLScene(PyObject *dict)
{
  PyObject *o;
  o = PyvtkMRMLScene_TypeNew();

  if (o && PyDict_SetItemString(dict, "vtkMRMLScene", o) != 0)
  {
    Py_DECREF(o);
  }
}
[...]

jcfr · August 29, 2020, 5:21pm

Thanks for following up with such a detailed answer, that is helpful.

The old “PythonD” libraries relied on the system loader to ensure that dependent libraries were loaded

For legacy reason, our custom wrapping infrastructure still generate the PythonD library for our custom VTK python modules. I will have to also address this.

Another possibility, though unlikely, is that multiple inheritance is gumming things up. Do your custom-wrapped objects use multiple inheritance?

Our custom classes are “classic” objects that only derives from vtkObject

dgobbi · August 29, 2020, 5:23pm

In that case, the problem is related to the handling of hierarchy files by your customized cmake scripts. Look at the .args file (response file) for MRMLCorePython to make sure that the --types option is set correctly.

E.g. when wrapping vtk-dicom, the cmake-generated .args file contains these:

--types '/work/vtk-dicom-build/lib/vtk/hierarchy/DICOM/vtkDICOM-hierarchy.txt'
--types '/work/vtk-build/lib/vtk/hierarchy/VTK/vtkCommonMisc-hierarchy.txt'
--types '/work/vtk-build/lib/vtk/hierarchy/VTK/vtkImagingCore-hierarchy.txt'
--types '/work/vtk-build/lib/vtk/hierarchy/VTK/vtkIOCore-hierarchy.txt'

jcfr · August 29, 2020, 5:33pm

Looks like we have the expected info:

// Libs/MRML/Core/MRMLCorePython.Debug.args

--hints "/home/jcfr/Projects/Slicer/Libs/MRML/Core/Wrapping/Tcl/hints"
--types "/home/jcfr/Projects/Slicer-Debug/Slicer-build/MRMLCoreHierarchy.txt"


-D"PYTHONQT_USE_RELEASE_PYTHON_FALLBACK"
[...]

-I"/home/jcfr/Projects/Slicer-Debug/Slicer-build/Libs/vtkTeem"
[...]

// /home/jcfr/Projects/Slicer-Debug/Slicer-build/MRMLCoreHierarchy.txt

[...]
vtkMRMLScene : vtkObject ; vtkMRMLScene.h ; MRMLCore
vtkMRMLScene::NodeReferencesType = std::map<std::string, std::set<std::string> > ; vtkMRMLScene.h ; MRMLCore
vtkMRMLScene::SceneEventType : enum ; vtkMRMLScene.h ; MRMLCore
vtkMRMLScene::StateType : enum ; vtkMRMLScene.h ; MRMLCore
vtkMRMLScene::Superclass = vtkObject ; vtkMRMLScene.h ; MRMLCore
[...]

Something is preventing the setting of is_vtkobject from working as expected …

I will make sure that the wrapping tool is invoked with the expected argument.

github.com

Slicer/VTK/blob/8ec81d403c4d87a2787fc3e03e2087d812d05d0b/Wrapping/Tools/vtkWrapPython.c#L427-L451


      
          /* guess whether type is a vtkobject */
          is_vtkobject = (data == file_info->MainClass ? 1 : 0);
          if (hinfo)
          {
            is_vtkobject = vtkWrap_IsTypeOf(hinfo, data->Name, "vtkObjectBase");
          }
          
          if (!is_vtkobject)
          {
            /* mark class as abstract only if it has pure virtual methods */
            /* (does not check for inherited pure virtual methods) */
            data->IsAbstract = 0;
            for (j = 0; j < data->NumberOfFunctions; j++)
            {
              FunctionInfo* func = data->Functions[j];
              if (func && func->IsPureVirtual)
              {
                data->IsAbstract = 1;
                break;
              }

This file has been truncated. show original

jcfr · August 29, 2020, 5:35pm

Should the --types for all VTK dependencies be listed in the response file ?

Instead of:

// Libs/MRML/Core/MRMLCorePython.Debug.args

--hints "/home/jcfr/Projects/Slicer/Libs/MRML/Core/Wrapping/Tcl/hints"
--types "/home/jcfr/Projects/Slicer-Debug/Slicer-build/MRMLCoreHierarchy.txt"

[...]

should I get something like

// Libs/MRML/Core/MRMLCorePython.Debug.args

--hints "/home/jcfr/Projects/Slicer/Libs/MRML/Core/Wrapping/Tcl/hints"
--types "/home/jcfr/Projects/Slicer-Debug/Slicer-build/MRMLCoreHierarchy.txt"
--types "/home/jcfr/Projects/Slicer-Debug/VTK-build/lib/vtk/hierarchy/VTK/vtkCommonMisc-hierarchy.txt"
...

[...]

dgobbi · August 29, 2020, 5:37pm

The following, by itself, is not sufficient because it only traces back to vtkObject:

vtkMRMLScene : vtkObject ; vtkMRMLScene.h ; MRMLCore

It is necessary to trace all the way back to vtkObjectBase, so the following must also be present in one of the hierarchy files provided via --types:

vtkObject : vtkObjectBase ; vtkObject.h ; vtkCommonCore

So those other hierarchy files are probably necessary.

jcfr · August 29, 2020, 5:41pm

That makes sense and is likely the issue. I will make sure the expected --types are listed and report back.

Thanks for the help

jcfr · August 29, 2020, 8:53pm

That was the issue. For future reference, see https://github.com/Slicer/Slicer/pull/5141/commits/473ef80b7f1f2fb055ff61e23417f0403f5f38cd

@dgobbi I also added you as co-author of the commit. Thanks again for the prompt help.

After “merging” the PythonD with the Python library, Slicer now builds and starts.

We are getting there

pieper · August 29, 2020, 8:56pm

Nice work @jcfr and thank you @dgobbi!