Building VTK on Windows

I have been trying to build VTK on Windows 10 with Python wrappings for the last week. I would like to add all dependencies that would make 99.99% of utilities VTK offers run as fast as possible. My system is:

Hardware

OS: Windows 10 Enterprise, x64-bit, Build 1909
CPU: 2x Intel(R) Xeon(R) Gold 6248R
Disk: 2TB NVMe M.2 SSD
RAM: 192 GB DDR4
Compute GPUs: 2x NVIDIA Quadro RTX8000 in TCC mode with NVLink
Display GPU: 1x NVIDIA Quadro RTX4000

Software

Visual Studio: [Visual Studio 16 2019 Community][1]
Windows SDK: [10.0.19041.0][2], targeting 10.0.18363
Python: [3.8.10 x64-bit][3]
CMake: [3.21.1][4] (I use CMake-GUI)
vcpkg: [2021-08-12][5]
VTK Source: [vtk-9.0.3-cp38-cp38-win_amd64][6] (downloaded from VTK website; not GitHub master)
VTKPythonPackage: [GitHub Link][7] (…not sure if I need this…)
CUDA Toolkit: [11.4][8]
TBB: [2021.3.0][9] (via Intel(R) oneAPI Base Toolkit)
MPI: [2021.3.0][9] (via Intel(R) oneAPI HPC Toolkit)

NOTE: I also have Embree, OSPRay, and OpenVKL installed via the Intel(R) oneAPI Rendering Toolkit

I have posted a detailed explanation of my efforts here on Stack Overflow: Build VTK with Python Bindings and CUDA, TBB, & MPI on Windows

After extensive discussions, I have realized that MPI and CUDA will probably not help me much at all, so we can ignore those libraries. I am using Python 3.8.10 and VTK 9.0.3, but if absolutely necessary, I am willing to using a 3.9 python and an older VTK 9 version.

Currently, using CMake GUI and the Visual Studio IDE for building is giving me trouble. I was hoping to use CMake GUI since there are so many options to configure with VTK and I wasn’t sure how many times I would need to reconfigure. Using the Visual Studio IDE, the build completes without errors, but I get far fewer files than if I run the following from the terminal:

> cd C:\VTK
> mkdir src
> mkdir build

then copy and unzip VTK-9.0.3.tar.gz from the VTK Downloads Page with 7-zip into src and run

> cd build
> cmake -DVTK_WRAP_PYTHON=ON -DVTK_PYTHON_VERSION=3 -DBUILD_SHARED_LIBS=ON ..\src
> msbuild.exe VTK.sln -property:Configuration=Release

If someone can please help me figure out the following, I would be grateful:

  1. How to include support for parallel processing, TBB, Embree, OSPRay, Qt, as well as recommendations and instructions for any and all other libraries to make “the majority” of VTK have 99.99% of its full possibilities and power.

  2. How to wrap Python. I would like to create a virtual environment and be able to import vtk with all this power.

Thank you!

I have been configuring and generating files into subfolder build and then building the solution in Visual Studio into the top-level C:\VTK folder so my final directory structure is this:

C:/VTK
|_ bin/
|_ build/
|_ include/
|_ lib/
|_ share/
|_ src/

Can someone confirm if I should actually be building the VTK.sln in the build folder if I have put my generated CMake GUI files in build? I’m going to be so embarrassed if this is my problem…

When you refer to building VTK.sln in C:\VTK, do you mean that you set the CMAKE_INSTALL_PREFIX to C:\VTK? If so, then that should be fine.

After cmake/CMakeGUI generates the .sln, the only further configuration that should be done in Visual Studio is to choose “Release” or “Debug” before performing the build. Any other changes in configuration should be done with cmake, not with Visual Studio.

If you want to optimize VTK’s speed then you should not try to make 99. 99% of VTK faster, but focus all your attention on improving the 1% that matters for your use case. You can use a profiler to identify performance bottlenecks.

2 Likes

@lassoan My only qualm with that is that means every time I have a new need/use case, I will need to rebuild VTK. There are several problems with this:

  1. It is very likely I will forget how the build process works at that point in the future. Since I have dedicated the time to figuring out the build process now, it makes sense to build as much as I can all at once.

  2. The build procedure is very likely to change in the future. Thus, what I will have build now won’t work any longer and I will have to rebuild everything, or there may be certain libraries that become incompatible with others.

  3. It is very likely libraries I have now will become unavailable (undownloadable) in the future. Therefore, if I write a giant application and with only several libraries I need for the moment right now, if I want to upgrade my build to include new features, I might not be able to.

You can expect that you (and others that are interested in your software) will often need to rebuild it from scratch, including all its dependencies. Therefore, it is probably worth automating the build process (for example, using CMake’s ExternalProject infrastructure) and then you don’t need to worry about rebuilds.

Optimizing build options, choosing optimal SMP backend, etc. may make your software a few ten percent, or maybe in extreme cases a few times faster on a specific hardware/software configuration (while potentially making your software slower or not run at all on other configurations). These improvements are dwarfed by the several magnitudes faster speed that you may achieve by addressing specific performance bottlenecks (by choosing optimal data structures, algorithms, processing parameters, using level-of-detail techniques, etc.).

How does this work and where is there documentation on this? In particular, I need instructions for Windows and I have not found anything in the VTK User’s Guide or Toolkit manuals.

Same question.

I am primarily interested in running tasks in parallel (i.e. parallel processes across cores and parallel threads). How can I enable and how do I use parallelization within VTK?

You can use build system of existing applications as examples:

There is no easy answer. You start by setting up a profiler and the next steps depends on what performance bottlenecks you find.

In general, using anything but the Sequential SMP backend should be sufficient. There are a few GPU-accelerated mappers in VTK (for volume rendering, glyphing, etc.), some custom rendering passes and shader replacements, and about 30 filters implemented in vtk-m that may give some extra performance boost if you have a strong GPU or have dozens of CPU cores - but you need to do profiling first to see if any of these features are relevant for what you want to do.

Excellent! Thank you for your reply! Is there a tutorial somewhere on profiling as I have not done this before.

I don’t know about any VTK-specific performance profiling tutorials. You can just search the web.

@mwestphal @ben.boeckel @dgobbi These kinds of things are problematic. Not having a single source of truth for how to use CMake, VTK, ParaView not only leads to misinformation depending on the resource found, but it makes it near impossible for new users to make just about any sense of these software packages.

@lassoan FYI, attempting to build the ParaView SuperBuild immediately errors with:

CMake Error at superbuild/cmake/SuperbuildUtils.cmake:45 (message):
  CMAKE_BUILD_TYPE must be one of: Release, RelWithDebInfo (found Debug).
Call Stack (most recent call first):
  superbuild/CMakeLists.txt:29 (include)

Also, it’s not downloading nearly as much with --recurse as the non-superbuild repo does, which leads me to believe it isn’t downloading everything.

You can get help for building an application on that application’s forum.

You can get help for building an application on that application’s forum.

Help/Support questions are supposed to go on the Discourse page. If I post an issue on the ParaView GitHub/GitLab page, the maintainers will kick it back.

There is no correpsonding ParaView SuperBuild Discourse page, so this is the right place to ask it.

However, there does seem to be a build issue with the superbuild, so I’ve added my issue here:

[Windows] Release configuration fails when running ninja install

Cc @ben.boeckel @mwestphal

As I wrote above, you can ask for help on the application’s forum (https://discourse.paraview.org).

@lassoan Ah I see. Sorry for misunderstanding. You meant to post on the ParaView Discourse page rather than here on the VTK Discourse page. My apologies.

I will post a question there and add a corresponding link here.

1 Like

@lassoan I have posted a question on ParaView Discourse here:

ParaView Superbuild Failing on Windows

If you can provide any help and/or forward this issue to others who can also help, I would greatly appreciate it.

Thank you!

CMake documents how to use CMake, VTK documents VTK, and ParaView documents ParaView. I can’t justify explaining advanced CMake usage in all 3 places (basics are fine, but anything in VTK/ParaView going into details about “how to set up your environment” for every platform is difficult to keep up-to-date reliably). If you could please describe:

  • What documentation did you read, if any? If none…well, how can we make what we do have more discoverable?
  • How it was the documentation deficient?

Did you read the README (or the error message?). Granted, the superbuild is not really intended for end users most of the time, so its docs are more terse.

Well, no. It downloads what it needs during the build. The superbuild is basically a repository of build instructions for many packages and logic for bundling it up into binaries.

Depends on the developer. If it’s worded as a support question, it’ll probably come here. If it’s worded as a concrete defect, it usually stays there.