explain the vtk architecture design

That would necessitate either vtkObject losing its internal refcount or using something like boost::intrusive_ptr. I suppose it could inherit from shared_from_this to get what it wants, but even that is unnecessary overhead I feel. But just migrating to shared_ptr would mean lots of API churn. The easy fix of passing shared_ptr around everywhere really sucks and is a really bad code smell and VTK isn’t all that good about indicating ownership or sharing through its API types (using T& return values to indicate “this is just a reference; you need to keep me around if you want to keep using that” and (const) T& as a parameter for “I just need to look; I won’t increment the refcount”).

2 Likes

thank you @mwestphal. Are there other open source c++ codes that do something different to get similar results with inhibiting leak detection? Is there an approach that can do this incrementally in a code base, i.e. baby steps?

@ben.boeckel can you unpack what the deeper problem would be with passing shared_ptr around?

This is great feedback so far.

While VTK has a long, successful, history, it’s C++ roots go back all the way to pre-C++98 code. Due to reluctance for wide, sweeping API updates for things, its code has never been modernized to use post-C++11 idioms pervasively (they get used here and there, but it’s mostly limited to what clang-tidy can fix in one fell swoop without many false positives). I’d look to VTK for workflow tips for a large project since that is really hard to get right over the long term, but I don’t know how much I’d use it as a model for modern C++. VTKm would be better if you’re doing lots of metaprogramming. CMake is becoming more modern over time, but still has old patterns laying around. I suggest using one of the guideline sets for modern C++ (Core Guidelines, Jason Turner’s C++ best practices, etc.).

2 Likes

I’d just use ASAN and/or Valgrind these days. I believe one of the reasons for VTK doing leak tracking is that its objects can tend to be quite large and leaking “just” one data array can actually mean you’re sitting on multiple megabytes of information. For VTK’s use cases, that isn’t really acceptable, so having an always-on runtime leak detector available is important.

1 Like

@ben.boeckel thank you. So what you are saying is don’t build in leak detection as it’s potentially not as high a priority depending on the application?

If ASAN (which requires build flags) is not sufficient for your use case, then I would look at build leak detection mechanisms. But, it could be a significant API redesign since you basically need to interpose in any allocation of your classes. But I’d be surprised if ASAN weren’t good enough to help track things down assuming other modern C++ practices (value semantics, const-correctness, etc.).

1 Like

Does anyone know the justification for headers and source being grouped together in VTK?

Why does VTK pair the cxx files with the header files in the directory structure. I see a lot of projects that use a separate include directory with the header. I think this is rather confusing, and like how VTK is organized, but Its unclear to me what the distinction is. I think is just a preference but I am curious to the benefits or downsides are to each approach.

Separating the headers into another directory is useful if you have lots of private headers that shouldn’t be visible to other users (they’d stay with the sources). VTK has maybe 5 private headers in its entire tree, so it isn’t worth it. Plus, VTK flattens all the headers in the install tree anyways. The distinction is basically one of trying to avoid accidentally including private headers from outside the library they are private to. Installation is also much easier since they can just be installed as a directory rather than as a file listing.

1 Like

Can someone point me to the design of how swig is being used in vtk? Or explain that succinctly?

That one is easy :slight_smile: . Swig is not used in VTK.

How do the python, java, and tick binding get built?

Tcl/Tk bindings no longer get built (they were removed in 2017). The other two are built on some programs VTK builds that read the VTK headers and write out code for Python and Java to use VTK objects. The generated sources are then tossed back at CMake to write out the rules for actually making the libraries and modules necessary. The tools themselves are pretty tailor-made to (or at least have sizeable exceptions and special cases for) VTK classes and its class hierarchy, so they’re not general-purpose wrapper generators. @dgobbi is the main person to ask for more details on the actual implementation of the tools.

1 Like

In very broad strokes, using only Python as an example, the program vtkWrapPython is executed for each VTK header file and generates Python bindings for that header file.

CMake is responsible for generating the build rules that execute vtkWrapPython and that compile the generated Python binding code into Python extension modules.

The vtkWrapPython program itself is very complex, with over 10000 lines of code split across 20 or so source files. It includes a yacc parser that understands most of the C++ language grammar. The same yacc parser is also used for the programs that generate the Java bindings.

1 Like

Wow! I didn’t realize that. What were the the reasons for a custom solution? versus say swig or something else? Does it then limit interfacing to c++, python and java? What about c#? Or maybe rust?

What are the other options out there?

C# wrapping of VTK is available in the commercial tool Activiz :
https://www.kitware.eu/product/activiz

It is also done with another dedicated tool.

VTK has supported wrapping since its very early days, I don’t know if swig had even been released back then. In any case, swig (or any other tool) wouldn’t be able to wrap VTK without a lot of help. Python is the only wrapping language that has really taken off for scientific computing, so it seems the best place to focus most of the effort.

What about matlab? (Only because customers )

Probably customers should switch to Python/numpy/scipy, too. Matlab’s popularity has been rapidly declining in the last few years. The business model of a single company providing all basic software components is just not competitive anymore: it costs too much, still too limited, and free&open alternatives are good enough or better. Overall, I would not worry about supporting Matlab and would not like to see that happen at the expense of making VTK’s Python support even better.

1 Like

I believe new language wrappers would be best served by creating a vtkWrapC setup and then using the normal FFI mechanisms from that language from there. It’d get us quite a big bang-for-the-buck on the VTK maintenance side for new language wrappers for sure. But that would also be a big undertaking in and of itself anyways, so seeing consumers of such a thing would almost certainly be necessary before it actually gets made.

As for Rust specifically, I’m not sure how well VTK’s API communicates lifetimes of things, so that would also entail lots of documentation on various APIs for how to indicate lifetimes.

I agree wholeheartedly. If VTK had a well-defined and introspectable C ABI, then the bindings of many languages (including Python) could be made to be very lightweight. Instead of wrapping every single VTK class for Python, we could just have some generic translation code that handled all classes. Still a very big project, though.

Speaking of MATLAB, many years ago I worked on MATLAB-Simulink wrappers for VTK and ITK. It was a fun project, but there was very little interest from the community. Fortunately I was able to fold some of the effort back into the VTK Python wrappers.

1 Like