Minimal VTK build with new module system

With vtk-8.1, Disabling groups and enabling only those modules needed was pretty straightforward to get only the desired pieces of vtk. Dependent modules of those requested were automagically also enabled.

I downloaded master and am confused by the new 5-state modules.
I turned off groups, and set the state of desired modules to ‘YES’. It seems dependent modules are no longer automatically enabled. For instance, I enabled CommonCore, IOGeometry but get CMake errors that IOGeoemtry requires CommonExecutionModel and CommonSystem. ( I didn’t need to explicitly enable them in vtk-8.1).

I started down the path of enabling the dependencies, but wonder if there is a better way to get behavior similar to 8.1, where I only need to request the higher level modules I know I need, without also needing to know about and enable all the dependent modules, too?

If you want dependencies enabled as needed, use DONT_WANT for those you don’t care about, not NO. The issue with the old system was that if you didn’t want a feature enabled, it was hard to ensure that was the case (extra dependencies or something) since dependencies got silently enabled. The NO state ensures that such modules are not built at all. That’s the error you’re encountering.

This is documented in the build documentation for VTK (suggestions on making that more discoverable is welcome).

Thank you, especially for the link.

I didn’t change any modules settings except those I wanted. I changed them to ‘Yes’. All other module settings appear to be ‘DEFAULT’. Are you saying I need to change all the other module settings to DONT_WANT? If so, that seems strange, because then I would also need to know about all the module names, and will make for a very large cmake command line when trying to script the build process.

DEFAULT is either WANT or DONT_WANT based on the VTK_BUILD_ALL_MODULES setting.

There are some settings which force modules off. For example, if you say VTK_WRAP_PYTHON=OFF, you will no longer get wrapping enabled just because you asked for the RenderingMatplotlib module. Other flags act similarly in that if they are off, the feature is not available. You may be running into that, but I don’t suspect so.

What’s the full set of flags you’re passing to the initial configure?

BUILD_SHARED_LIBS=true
VTK_BUILD_TESTING=false
VTK_BUILD_DOCUMENTATION=false
VTK_REPORT_OPENGL_ERRORS=true
VTK_ALL_NEW_OBJECT_FACTORY=true

VTK_GROUP_ENABLE_Imaging=NO
VTK_GROUP_ENABLE_MPI=NO
VTK_GROUP_ENABLE_Qt=NO
VTK_GROUP_ENABLE_Rendering=NO
VTK_GROUP_ENABLE_StandAlone=NO
VTK_GROUP_ENABLE_Views=NO
VTK_GROUP_ENABLE_Web=NO

VTK_MODULE_ENABLE_CommonCore=YES
VTK_MODULE_ENABLE_FiltersFlowPaths=YES
VTK_MODULE_ENABLE_FiltersHybrid=YES
VTK_MODULE_ENABLE_FiltersModeling=YES
VTK_MODULE_ENABLE_GeovisCore=YES
VTK_MODULE_ENABLE_IOEnSight=YES
VTK_MODULE_ENABLE_IOGeometry=YES
VTK_MODULE_ENABLE_IOLegacy=YES
VTK_MODULE_ENABLE_Module_vtkIOPLY=YES
VTK_MODULE_ENABLE_IOXML=YES
VTK_MODULE_ENABLE_InteractionStyle=YES
VTK_MODULE_ENABLE_RenderingAnnotation=YES
VTK_MODULE_ENABLE_RenderingFreeType=YES
VTK_MODULE_ENABLE_RenderingOpenGL2=YES
VTK_MODULE_ENABLE_libxml2=YES

@ben.boeckel this 5-state system is really confusing. It would be so much better to have just ON/OFF state as in every other CMake project. When there is a dependency issue, you can show an error message to the user, such as “X module is set to ON but required Y, W, Z modules are set to OFF. Please set module X to OFF or set Y, W, Z modules to ON.” This works well for us in other library where we have complex dependencies between dozens of components.

When you first configure the project, you may get a few of these questions, but these questions are very valuable: they inform developers about decisions that they have to make and allow them to decide for themselves. It is a very simple, transparent mechanism. Answering a few important questions is much less annoying than battling with some automatic rule engine, trying to reverse-engineer the hidden dependencies and guess what could be the impact of changing the state of a module (which may enable or disable a whole set of other modules without knowledge or consent of the developer).

With the above settings, the first error I see is this:
CMake Error at CMake/vtkModule.cmake:953 (message):
The VTK::IOGeometry module requires the disabled module
VTK::CommonExecutionModel.
Call Stack (most recent call first):
CMakeLists.txt:253 (vtk_module_scan)

If I proceed to enable each disabled module (by changing its state to YES) as they are called out by the cmake process, I end up having to enable all kinds of modules I do not want, especially parallel modules.

Am I missing something important with the new process?

Use DONT_WANT here. The values for groups is the same as modules.

I don’t disagree that it can be confusing. However, I suspect that you’re underestimating the number of reconfigures you’re going to need to flush out the dependency errors. With only 2 states, you’ll basically have to manually enable the entire dep tree for what you want (or we lose the way to say the valuable “this build must not have Python” (or MPI, or other feature) configuration).

The better solution for this is, IMO, to split VTK into smaller projects that are more focused. They can live in the same repo, but a single build with 100+ optional libraries is never going to be easy to configure.

2 Likes

Yes please!

I fully agree that stripping down VTK core and moving out most modules to separate projects, (which could be individually hosted, developed, built, and Python-wrapped) would solve many issues. If a number of VTK modules would be spun off to separate projects, but still maintained by Kitware, then that would ensure that a good infrastructure is developed and maintained (such as convenience CMake scripts in VTK core), which the community could use for creating their own VTK modules.

Thank you for the guidance. I started a fresh build attempt and used ‘DONT_WANT’ for the group enable values and set the module values to ‘YES’ for those I wanted. I am still getting errors regarding disabled modules. The modules tagged as missing have values of ‘DEFAULT’
Should I be using ‘WANT’ instead of ‘YES’?

I only get 1 error per invocation of cmake’s configure, so its a slow process working through the errors. Here are the first three I saw:

CMake Error at CMake/vtkModule.cmake:2343 (message):
  The VTK::RenderingContext2D dependency is missing for VTK::ChartsCore.

CMake Error at CMake/vtkModule.cmake:2343 (message):
  The VTK::octree dependency is missing for VTK::RenderingLabel.

CMake Error at CMake/vtkModule.cmake:2343 (message):
  The VTK::octree dependency is missing for VTK::RenderingLabel.

Passing -DVTK_DEBUG_MODULE=all will output more debugging information about why those modules aren’t available.

Note that this vastly increases our release cycle burden and will mean that VTK cannot break API without announcement and coordinated changesets. That’s going to be the tough pill to swallow for the development community because we are, on the whole, very fond of breaking API. The split-up-VTK thing is a goal I’d like to have, but I don’t think it’s anywhere close to being realized today. I have enough other things taking up my time, but adopting some of the more domain-specific modules into separate repositories would be a good first step others can take in a piecemeal way.

Here’s the VTK_DEBUG_MODULE output specifically related to the modules called out in the error message:

VTK module debug module: VTK::ChartsCore declared by C:/vtk-master/Charts/Core/vtk.module
VTK module debug enable: VTK::ChartsCore is DEFAULT, using group `StandAlone` setting: DONT_WANT
VTK module debug provide: VTK::ChartsCore is indeterminite (DONT_WANT)
VTK module debug provide: VTK::ChartsCore is provided due to dependency from VTK::CommonDataModel

VTK module debug module: VTK::RenderingContext2D declared by C:/vtk-master/Rendering/Context2D/vtk.module
VTK module debug enable: VTK::RenderingContext2D is DEFAULT, using group `Rendering` setting: DONT_WANT

VTK module debug kit: VTK::RenderingContext2D belongs to the VTK::Rendering kit

CMake Error at CMake/vtkModule.cmake:2343 (message):
  The VTK::RenderingContext2D dependency is missing for VTK::ChartsCore.
Call Stack (most recent call first):
  CMakeLists.txt:315 (vtk_module_build)

What ends up requiring VTK::ChartsCore? Is the topo sort wrong? Is it enabled by a test dependency (in which case, you’re hitting https://gitlab.kitware.com/vtk/vtk/issues/17509)?

The trigger appears to be the GeovisCore module. I removed it from my list of enabled modules in a fresh build and did not get the error. I’m using CMake-gui on WIndows 10. Is there a log file generated that I can upload? I tried to cut-n-paste just the module debug parts of cmake output but ran into the word limit.

Discovered the error on my part based on your comment regarding test dependency.
I was trying to turn off testing by setting BUILD_TESTING (old name for the var).
I fixed it by using VTK_BUILD_TESTING at the same time as I removed GeovisCore module from the build. Sorry for the red herring.

I believe I have the situation under control now, by turning off VTK_BUILD_TESTING, using DONT_WANT to turn off groups, and YES to turn on modules.

Thank you for your patience in helping me learn the new module system.

@ben.boeckel how can we know the value of DEFAULT for a given module?

See the “Enabling modules for build” section in the module system’s documentation.