CMake flags used to compile Python wheel

Support
1

Hi,
I was trying to go through the .ci/*.cmake files to understand how exactly the Python3 wheels (mainly interested in Linux) are generated, but I found that many optional flags are not mentioned (thus I’m assuming set to off?). I’m mainly looking at VTK_PYTHON_FULL_THREADSAFE (that I’m assuming is off) and VTK_SMP_IMPLEMENTATION_TYPE (also sequential?). I have been wondering why, considering how hard it is to achieve multithreading in Python due to the GIL, these flags are not turned on by default to have multithreaded code on the C++ side of things.

Francesco

2

Oddly enough, the VTK_PYTHON_FULL_THREADSAFE setting can only be configured when building Paraview. It doesn’t appear when building VTK itself.

I agree that VTK_PYTHON_FULL_THREADSAFE should be ON by default, and last year I submitted a patch (!9223) to VTK. The same patch also releases the GIL during long stretches of C++ code execution, in order to unblock the execution of python threads. If you’re willing to read through the patch and give your opinion, that might help to give it enough momentum to actually get fully reviewed and merged.

Regarding C++ threading, the wheels provide both Sequential and STDThreads as SMP backends. The VTK_SMP_IMPLEMENTATION_TYPE only indicates the default backend. The full set of defaults for VTK (and the wheels) is as follows:

VTK_SMP_IMPLEMENTATION_TYPE=Sequential
VTK_SMP_ENABLE_SEQUENTIAL=ON
VTK_SMP_ENABLE_STDTHREAD=ON
VTK_SMP_ENABLE_TBB=OFF
VTK_SMP_ENABLE_OPENMP=OFF

The STDThread backend can be turned on at runtime like so:

>>> vtkSMPTools.SetBackend('STDThread')

It isn’t the default because it has performance issues (and potential stability issues) which are being worked on in !10027.

The VTK imaging pipeline, at least, is parallel on all platforms/configurations because it falls back to native threads (e.g. posix threads on linux).