Using VTK shared libraries in (CERN) ROOT

Support
1

I have recently (quite accidentally) got interested in VTK.

I am trying to use its shared libraries from inside of ROOT.

The current status is that I can dynamically load VTK libraries in a ROOT interactive session and I can run “Examples/Tutorial/Step*/Cxx/Cone*.cxx” as interpreted C++ macros.

Unfortunately, nothing is displayed on the screen.

It seems that VTK plays some dirty tricks, hidden from a “casual” user, which perform the actual initialization of shared libraries at startup.

I failed to “mimic” this behavior.
I tried to inject (into the ROOT’s C++ interpreter):

root [0] #include "vtkAutoInit.h"
root [1] VTK_MODULE_INIT(vtkRenderingOpenGL2);
ROOT_prompt_1:1:1: error: namespaces can only be defined in global or namespace scope
VTK_MODULE_INIT(vtkRenderingOpenGL2);
^
/.../include/vtk-9.0/vtkAutoInit.h:88:3: note: expanded from macro 'VTK_MODULE_INIT'
  {                                                                                                \
  ^
root [2]

Could you, please, give me a hand with this. Thanks in advance.

2

Hello, Wile,

First, I am a great fan of CERN’s work. Congratulations !
As for the error you’re getting in ROOT’s C++ parser, if you open the offending macro definion you find:

#define VTK_MODULE_INIT(M) \
  VTK_AUTOINIT_DECLARE(M) \
  static struct M##_ModuleInit {                                           \
    /* Call <mod>_AutoInit_Construct during initialization.  */            \
    M##_ModuleInit()  { VTK_AUTOINIT_CONSTRUCT(M) }                      \
    /* Call <mod>_AutoInit_Destruct during finalization.  */               \
    ~M##_ModuleInit() { VTK_AUTOINIT_DESTRUCT(M)  }                      \
  } M##_ModuleInit_Instance;

Notice that it is defining a class and creating a static instance of such class in tandem. That means that such code is supposedly to run before main(). This is a common resource used to initialize plug-in DLLs/SOs in extensible software. So, the question is: Is ROOT’s parser able to create static variables?

regards,

Paulo

3

Creating a static instance of some class should not be a problem.

The ROOT’s C++ interpreter complains about the “namespace”. Maybe its preprocessor has problems with parsing the involved macros?

But my first problem is … is calling “VTK_MODULE_INIT(vtkRenderingOpenGL2);” the right thing to do, or is it not?
If yes, is it sufficient, or do I need to do something more?

4

If you’re using CMake, I would recommend to use the vtk_module_autoinit command to do all of this dirty work for you. There’s no need for anyone to have to relearn how autoinit works just to use VTK unless they just really don’t want to use CMake (unless someone wants to know how to make static registration work robustly across all of the worst combinations of platform/linker/runtime loader behaviors).

5

CMake will / can not be used.
I need to write a C++ interpreted macro which will “inject” the required “magic” into the ROOT’s C++ interpreter (that’s also the way I dynamically link VTK shared libraries on demand).
In principle, Linux and possibly macOS are my primary targets (same as what ROOT supports).

BTW. Is there somewhere any “detailed” technical description of how VTK initializes its modules?

6

Yes… but keep that in mind. Can you try this in the parser?

static struct Foo {
public:
   Foo(){}
   ~Foo(){}
} foo;

I also suspect of some ill-supported preprocessor language feature. VTK_MODULE_INIT unfolds into other macros, that unfolds into other macros themselves.

Yes. Like you, I don’t use CMake. Non-CMake users must include those macros explicitly. In the .cpp of my VTK widget class, I put them in the very beginning of the source file:

#include <vtkAutoInit.h>
VTK_MODULE_INIT(vtkRenderingOpenGL2) // VTK was built with vtkRenderingOpenGL2
VTK_MODULE_INIT(vtkInteractionStyle)
VTK_MODULE_INIT(vtkRenderingFreeType)

It depends on your needs (mine requires two others). But, again, getting a namespace error suggests some unsupported syntax. An incorrect module initialization would result in a blank screen like you had, a VTK error console to pop-up or even a crash.

From the documentation of vtkAutoInit.h:

This call must be made in global scope in the translation unit of your executable (which can include a shared library, but will not work as expected in a static library).

A “translation unit”, in a nutshell, is a .cpp file with all the contents of the headers, macros, autos, templates, etc. resolved into actual C++ code. That is, a source code ready to be compiled into an object file.

Despite all that, Python and Java users are able to have the VTK modules initialized in a interpreted context. So, we know it’s doable.

My first attempt would be adding the initialization of the required VTK modules to the ROOT’s parser code itself. Is it practical for you?

7

Oh, I missed that there was an interpreter/runtime loading going on here. The way it works for Python/Java is that the TU that registers the types to the VM/interpreter contains the autoinit magic, so that’s how they work with simple imports. Depending on how the interpreter works, I would have those calls in the glue between vtkModuleName.so and whatever the interpreter has. If the interpreter loads things manually, then there’s some manual work that needs to be done. The easiest thing to do (if it works) would be to load an implementation of any library containing registration targets (e.g., loading vtkRenderingCore would mean ROOT would load vtkRenderingOpenGL2 as well). However, this requires knowledge of what implements what and what set of modules handles the entire necessary set which is…not trivial.

1 Like
8

After linking to VTK librarires, you can try calling the initialization directly in the script console:

#include <vtkRenderingOpenGL2Module.h>
#include <vtkAutoInit.h>

// Run this just once!  Comment out this after the first run.
vtkRenderingOpenGL2_AutoInit_Construct();

//do your stuff

// ATTENTION: Uncomment this and run **before** closing ROOT.
// Failing to do so may result in OpenGL resources not being freed, requiring a system reboot.
// vtkRenderingOpenGL2_AutoInit_Destruct();

See if that works. The awkward way to initialize and free resources is to try to mimic the behavior of the static instance.

9

Many thanks for your replies.

I created a “C++ preprocessor problem” report on the ROOT Forum. I assume, in a couple of days, we will get a statement from the ROOT Team.

As I said, there should be no big problems with static singletons:

root [0] static struct Foo {
root (cont'ed, cancel with .@) [1]public:
root (cont'ed, cancel with .@) [2]   Foo(){}
root (cont'ed, cancel with .@) [3]   ~Foo(){}
root (cont'ed, cancel with .@) [4]} foo;
root [5] .class Foo
===========================================================================
struct Foo
SIZE: 1 FILE: ROOT_prompt_0 LINE: 1
List of member variables --------------------------------------------------
List of member functions :---------------------------------------------------
filename     line:size busy function type and name
(compiled)     (NA):(NA) 0 public: Foo();
(compiled)     (NA):(NA) 0 public: ~Foo() noexcept;
root [6]
10

When it comes to “vtkRenderingCore” and “vtkRenderingOpenGL2” …
Running “ldd libvtkRenderingOpenGL2-9.0.so” returns (among others) “libvtkRenderingCore-9.0.so.1”. That means that the “vtkRenderingCore” library will automatically be loaded when one loads “vtkRenderingOpenGL2”.

On the other hand, neither “vtkRenderingCore” nor “vtkRenderingOpenGL2” depend explicitly on “vtkInteractionStyle” nor “vtkRenderingFreeType”.

So, I will need to collect a list of “automatically” loaded libraries that need “manual” initialization afterward.

Can I assume that every VTK module provides a “void SomeModule_AutoInit_Construct()” function?
Of course, it can be completely “dummy”, if a module does not need any special initialization.
If not, how can I learn which modules need it and which do not?

11

Right, but for correctness, if vtkRenderingCore is loaded, you need to know to load vtkRenderingOpenGL2 to make its abstract classes actually work. The Python bindings have the same thing: if one imports just RenderingCore, some classes may not work without importing the RenderingOpenGL2 module to provide implementations.

No, only modules which implement others will have such a function.

12

Many thanks again.

Let me try a “real-life example”.

Looking into the “CMakeLists.txt” files for all “VTK-9.0.x/Examples/Tutorial/Step*/Cxx/Cone*.cxx” (yes, I know that they currently cannot be built by VTK, but they seem fine for my purposes), I can see the following list of modules:

CommonCore
FiltersSources
InteractionStyle
InteractionWidgets -> needed by "Step6/Cxx/Cone6.cxx" only
RenderingOpenGL2

How can I learn which of these modules needs initialization and which does not?
I assume the order of initialization may be important, too. How can I learn in which order they need to be initialized (e.g., can I assume that I need to do it in the same order in which they appear in the original “CMakeLists.txt” files)?

Neither the “vtkRenderingCore” nor the “vtkRenderingFreeType” are explicitly listed there but, from your previous posts, I assume they need initialization, too (as the “RenderingOpenGL2” is used).
How can I learn which “hidden” modules need initialization (and in which order)?

13

That information is stored as a property on the CMake target (_vtk_module_implements and _vtk_module_implementable), so probably not all that accessible to you either where you need it. The order doesn’t matter too much as it is rare that there is more than one implementation at any given time (though there are cases such as RenderingOpenVR; in this case, the first one registered “wins” (see this issue)).

Those are only needed if interaction or font rendering is needed.

14

It seems that you got a solution in ROOT’s forum. Did that work for you? If so, please, post the solution here so others in the future with a problem like youres can benefit from it.

15

Three more questions …

Do modules provide a function which returns, e.g., “true” if they need “initialization”?
I could implement something like this (just a “pseudocode”): “if (SomeModule_Needs_AutoInit()) VTK_MODULE_INIT(SomeModule);”

I looked into the “vtk.module” files of several modules.
They sometimes contain “IMPLEMENTABLE”, sometimes “IMPLEMENTS”, and sometimes both.
Do I need to “initialize” a module … when the “IMPLEMENTABLE” is present or when the “IMPLEMENTS” is present or in both cases or only if both are present?

Assume I have a list of modules, and I loop over this list, loading the corresponding shared libraries.
Should I “initialize” a module inside this loop as soon as the corresponding shared library is loaded?
Or should I implement an additional loop that will “initialize” modules after all shared libraries were loaded (in the first loop)?
Or maybe it does not matter if it is done in one common or two separate loops?

16

@Paulo_Carvalho Yes, I am able to run all 6 tutorials now. I am still working on it, though (see my previous post). When I’m ready, I will share the “solution”, of course.

17

No.

IMPLEMENTABLE means “has a factory which can be implemented” and IMPLEMENTS lists the modules for which the current module has implementations of. IMPLEMENTABLE is there as a sanity check to avoid generating calls to modules which have nothing to register to.

Only the order matters (since it is first-come-first-served for the default). The number of loops or whatever is unknown to the registration logic.

18

@ben.boeckel Many thanks. I will try the following approach then. For every VTK “component”, that I explicitly “load”, I will try to find if an appropriate “vtk*_AutoInit_Construct” entry exists in the loaded shared libraries and, if yes, call VTK_MODULE_INIT for this “component”.

@Paulo_Carvalho In one of your previous posts, you write that, if one calls “vtkRenderingOpenGL2_AutoInit_Construct();”, one is also expected to call “vtkRenderingOpenGL2_AutoInit_Destruct();”.
However, the “VTK_MODULE_INIT(vtkRenderingOpenGL2);” does NOT do it (there is no destructor).
Moreover, I “scanned” all available shared libraries, and I cannot find any function which name contains “_AutoInit_Destruct”.

19

Yes, there used to be dtor calls, but they were empty functions, so they were just removed. The problem is that the registrations hold pointers into the loaded code and “taking them back” is not generally possible, so unloading a VTK library is not generally possible so instead process teardown is used to undo such registrations.

20

It does. Take a look at one of my first posts. I posted the definition of VTK_MODULE_INIT. The function with the tilde (~) is the destructor. Leaving out the destructor is up to you, but it is considered bad practice.