VTK import is confusing in VTK tests

[lassoan (Andras Lasso)]

VTK tests have been recently reworked to improve robustness of VTK test execution. Unfortunately, this change has also made importing VTK into Python extremely complicated. This would be OK if only VTK core developers would be impacted, but it is causing confusion among users.

Users are now just blindly copying dozens of lines of incomprehensible nonsense from the VTK tests into their own code. See for example in this recent post by a VTK user:

# noinspection PyUnresolvedReferences
import vtkmodules.vtkInteractionStyle
# noinspection PyUnresolvedReferences
import vtkmodules.vtkRenderingOpenGL2
from vtkmodules.vtkCommonColor import vtkNamedColors
from vtkmodules.vtkCommonCore import (
    VTK_DOUBLE_MAX,
    vtkPoints
)
from vtkmodules.vtkCommonCore import (
    VTK_VERSION_NUMBER,
    vtkVersion
)
from vtkmodules.vtkCommonDataModel import (
    vtkIterativeClosestPointTransform,
    vtkPolyData
)
from vtkmodules.vtkCommonTransforms import (
    vtkLandmarkTransform,
    vtkTransform
)
from vtkmodules.vtkFiltersGeneral import (
    vtkOBBTree,
    vtkTransformPolyDataFilter
)
from vtkmodules.vtkFiltersModeling import vtkHausdorffDistancePointSetFilter
from vtkmodules.vtkIOGeometry import vtkSTLReader
from vtkmodules.vtkInteractionWidgets import (
    vtkCameraOrientationWidget,
    vtkOrientationMarkerWidget
)
from vtkmodules.vtkRenderingAnnotation import vtkAxesActor
from vtkmodules.vtkRenderingCore import (
    vtkActor,
    vtkDataSetMapper,
    vtkRenderWindow,
    vtkRenderWindowInteractor,
    vtkRenderer
)

Since users don’t understand what those lines are for and removing them “breaks VTK”, they will keep copying it in their code, making it longer, more complex, and more brittle (internal VTK reorganization may break their code).

For me the obvious solution would be to return to import VTK in tests the same way as we want users to import VTK. It would the simpler to implement and maintain by VTK developers and would be simpler for VTK users. If this is impossible then maybe warnings could be added around those special import statements, explaining to users that they should not copy-paste those lines into their code and what they should do instead.

@dgobbi @ben.boeckel

[will.schroeder]

Well said.

[dgobbi (David Gobbi)]

First, I’d just like to reitererate that the simple “import vtk” will continue to work. And I can remove wording in the VTK documentation that calls it “old” or “obsolete”.

I did read the post that you linked, and the user didn’t make any indication that they were confused by the imports. They were fairly well organized apart from the extra ‘vtkCommonCore’ with the unused items. It’s true that the imports are verbose, but they’re hardly incomprehensible.

Using this import style in the VTK tests actually helps to guard against the kind of module reorganizations that cause brittleness. Movement of nearly any class from one module to another will now be caught by the testing. Another advantage for the tests is that it isolates them to just the VTK libraries that they are testing.

But overall, I think people should use the style that best suits their needs. The VTK examples site was perhaps too eager to convert all the code to ‘vtkmodules’ imports, and it would be nice to see more variety there, especially since ‘import vtk’ is by far the most common style in the wild. I don’t think users will be overly confused by different examples having different styles.

[lassoan (Andras Lasso)]

@dgobbi

First, I’d just like to reitererate that the simple “import vtk” will continue to work. And I can remove wording in the VTK documentation that calls it “old” or “obsolete”.

Thanks for confirming, these are the most important. Still, I think we can make some more improvements in how we educate VTK users.

@dgobbi

I did read the post that you linked, and the user didn’t make any indication that they were confused by the imports.

The confusion is indicated by the fact that 20% of the code that the user posted was import statements. This is not normal and not a practice that VTK users should follow.

@dgobbi

Using this import style in the VTK tests actually helps

It would be nice to find a way to allow VTK core developers to benefit without impacting users. Adding a comment everywhere that this import style is required for VTK internal testing purposes and provide a link that describes alternatives would be one relatively easy way to improve the situation.

@dgobbi

The VTK examples site was perhaps too eager to convert all the code to ‘vtkmodules’ imports,

Thanks for pointing this out. Indeed, Python examples on the VTK examples website 10-20% of the source lines are import statements, which is pretty bad, especially since the only goal of the website is to educate users on how to use VTK.

So, I fully agree that we should see some variety there. Importing specific classes could be demonstrated in a few cases where it makes sense, but usually an example uses a lot of VTK classes, so using a single import vtk is much simpler.

@amaclean What was the motivation for switching to those long import statements in VTK examples?

[amaclean (Andrew Maclean)]

The motivation is simple … only include/import classes you actually need.

VTK is huge and well organised into modules. This approach in the VTK Python examples gives a good cross reference for C++ etc. where you only include the necessary classes. Indeed the user can see what modules are actually used. Yes it is a little wordy.

Here is a hint:
There is nothing stopping you using import vtk when initially developing, and then using VTKImportsForPython (which was specially developed for this purpose) to scan your code so that only the relevant classes are used.

[lassoan (Andras Lasso)]

In C++ it is important to minimize imports to avoid unnecessary dependencies and reduce recompilation time. There are no such concerns in Python. There could be performance issues with importing a large Python package, but import vtk takes only a fraction of a second, so that should not be a problem.

Since the lengthy imports don’t seem to have any perceptible advantages but they have clear disadvantages, and tests in VTK core don’t use import vtk anymore, it would be important to re-introduce the simple import vtk into VTK examples. @amaclean please give this a thought.

[lassoan (Andras Lasso)]

I did some more tests and importing selected VTK packages have some significant speed advantage in a special case: when importing VTK the first time after restarting the computer (cold start). Loading a single VTK module takes couple of seconds while import vtk can take 10-20 seconds. Therefore, we should preserve some examples of how to do imports of specific VTK modules.

[ben.boeckel (Ben Boeckel (Kitware))]

Another detail is that import vtk can fail if VTK is built with something “obscure” (say the OpenVR support) and it is not in the library path even if the importer doesn’t care about it.

[Jens_Munk_Hansen]

As a old fart linux programmer using vim/emacs with gtags etc, I am really happy with new approach. I tend to find IDEs slow for development and use some home-made trace tools for dumping a usefull stack rather debug mode.

First of all speed, but it is also quite neat that I can see what is included and verify correct behavior in python. Secondly, I can use a small script for converting samples from C++ to Python.

I do agree that it is probably not the best to make newcomers use VTK. Perhaps, a script could be made for developers such that the examples appearing on the website appears using “import vtk”.

[amaclean (Andrew Maclean)]

@Jens_Munk_Hansen regarding

@Jens_Munk_Hansen

converting samples from C++ to Python

It works really well the other way too in that you don’t have to work through the python code working out what includes are needed, also, since the modules are specified, you then know what to add to your CMakeLists.txt file for VTK COMPONENTS.

Addidtionally, from an earlier post:

@amaclean

Here is a hint:
There is nothing stopping you using import vtk when initially developing, and then using VTKImportsForPython (which was specially developed for this purpose) to scan your code so that only the relevant classes are used.