no libvtkRenderingTk with pip install vtk

itan · April 22, 2021, 2:16pm

Hi, I am on ubuntu 20.04 and trying to build a small GUI with Python 3.8.5, VTK 9.0.1 and Tkinter. Using VTK and Tkinter separately both work well, but I am having difficulties combining them.

I have tried it with a vtkTkRenderWindowInteractor and a vtkTkRenderWidget so far, but I obtain the error:

_tkinter.TclError: couldn't load file "libvtkRenderingTk-9.0.so": libvtkRenderingTk-9.0.so: cannot open shared object file: No such file or directory

I installed vtk via pip install vtk in my virtual environment. If I look inside the env/lib/python3.8/site-packages/vtkmodules there is a tk folder with the python files defining the classes I used. But there is indeed no libvtkRenderingTk.

Is it necessary to build vtk myself to achieve this? I guess with VTK_WRAP_PYTHON=ON and VTK_USE_TK=ON? I expected the pip installation to install all python modules and have all python functionality.

ben.boeckel · April 22, 2021, 2:21pm

The pip wheels only contain what doesn’t require external dependencies. I guess Tk is probably fine since Python usually ships with it. Not sure how Tk-less Python builds will handle the wheel then. If the auditwheel machinery doesn’t complain, we can enable that module for future wheel builds.

You can compile your own wheels by following these steps.

itan · April 22, 2021, 2:56pm

Great, thank you for the quick reply! I was not aware of that. I will try those instructions.

dgobbi · April 22, 2021, 3:09pm

Yes, VTK_WRAP_PYTHON ,VTK_USE_TK, and BUILD_SHARED_LIBS are enough to build libvtkRenderingTk-9.0.so

But even when it’s built, it still must be installed in the correct location before it can be used. CMake puts it into “libs”, but it isn’t just a library. It’s a Tk extension module. If it is placed inside python’s site-packages directory, then the vtkTkRenderWidget should be able to find it.

It might be possible to put it in the pypi wheel. As Ben said, Python includes the Tk run-time libraries. The trick is how to build it, since Python distros do not include all the Tk headers and other pieces needed for the build. So it’s easy to build on linux (just install tk-dev beforehand), but it can be very challenging on Windows.

I’ve attached a simple cone example that you can use for testing:
vtkRenderWidgetConeExample.py (1.3 KB)

Edit: since nothing links to or attempts to load libvtkRenderingTk.so until vtkTkRenderWidget is imported, there’s no harm in including it with Tk-less Python. It won’t interfere with anything else.

ben.boeckel · April 22, 2021, 4:07pm

Note that import vtk imports all of VTK, so the “easy” import will hit this problem.

dgobbi · April 22, 2021, 4:31pm

It actually won’t, since “import vtk” does not import vtkmodules.tk.vtkTkRenderWidget.

The vtkRenderingTk lib isn’t a Python extension module and it isn’t loaded like one. It’s loaded into Tk by the machinery in vtkmodules.tk.vtkLoadPythonTkWidgets. For convenience, vtkLoadPythonTkWidgets looks for vtkRenderingTk in sys.path, but only because the code was written that way.

ben.boeckel · April 22, 2021, 5:16pm

Ah, I forgot that the Tk module is quite an oddball .

nicoco · February 2, 2022, 11:31am

Hey all,

Thanks to this thread I managed to use VTK with tkinter and it’s great. However, I have to set LD_LIBRARY_PATH=/path/to/my/env/lib/pythonX.X/site-packages/vtkmodules before running my tkinter app. It is a minor inconvenience, but was wondering if there was a ‘cleaner’ way to set this up?

I tried this:

import os
import site
os.environ["LD_LIBRARY_PATH"] = os.path.join(site.getsitepackages()[0], "vtkmodules")

at the very top of my python entrypoint, but for some reason it does not work.

Any advice on how I could make this work, preferably in a cross-platform-friendly way?

ben.boeckel · February 2, 2022, 1:49pm

Adding an RPATH of $ORIGIN would suffice. I think this is fixed in master, but isn’t in any release yet.

dgobbi · February 2, 2022, 1:56pm

Hi Nicoco,

Try playing around with the pathlist in vtkLoadPythonTkWidgets.py to see if you can get it to load without setting LD_LIBRARY_PATH.

You can also try moving libvtkRenderingTk*.so from vtkmodules out to the same directory as libvtkCommonCore*.so* and the other VTK shared libraries.

The libvtkRenderingTk*.so is actually not a Python extension module, it’s a Tk extension module (which is why Python uses tkinter to load it). So it’s an odd beast.

You can also experiment with loading it by hand:

import vtkmodules.all as vtk
import tkinter
root = tkinter.Tk()
root.tk.call('load', ...)

where the tcl “load” command is documented here.

Out of curiosity, what Linux distro are you using?

David

nicoco · February 2, 2022, 3:08pm

Thanks for your answers!

I realized I did not include the error message in my first post

    self.vtk_render_window_interactor = vtkTkRenderWindowInteractor(
  File "/home/nicoco/.cache/pypoetry/virtualenvs/xxx-research-r1WBkZBg-py3.9/lib/python3.9/site-packages/vtkmodules/tk/vtkTkRenderWindowInteractor.py", line 56, in __init__
    vtkLoadPythonTkWidgets(master.tk)
  File "/home/nicoco/.cache/pypoetry/virtualenvs/xxx-research-r1WBkZBg-py3.9/lib/python3.9/site-packages/vtkmodules/tk/vtkLoadPythonTkWidgets.py", line 87, in vtkLoadPythonTkWidgets
    interp.call('load', filename, pkgname)
_tkinter.TclError: couldn't load file "libvtkRenderingTk-9.1.so": libvtkRenderingTk-9.1.so: cannot open shared object file: No such file or directory

Well… I have no idea what this means

Looking at this file, I realized that sys.path is involved, so I tried…

import os
import site
import sys
sys.path.append(os.path.join(site.getsitepackages()[0], "vtkmodules"))

…and it worked!

They are already in the same dir:

nicoco@licorne ~/s/i/i/research (master)> 
fdfind libvtkCommonCore /home/nicoco/.cache/pypoetry/virtualenvs/xxx-research-r1WBkZBg-py3.9
/home/nicoco/.cache/pypoetry/virtualenvs/xxx-research-r1WBkZBg-py3.9/lib/python3.9/site-packages/vtkmodules/libvtkCommonCore-9.1.so
/home/nicoco/.cache/pypoetry/virtualenvs/xxx-research-r1WBkZBg-py3.9/lib/python3.9/site-packages/vtkmodules/libvtkCommonCore.so
nicoco@licorne ~/s/i/i/research (master)> 
fdfind libvtkRenderingTk /home/nicoco/.cache/pypoetry/virtualenvs/xxx-research-r1WBkZBg-py3.9
/home/nicoco/.cache/pypoetry/virtualenvs/xxx-research-r1WBkZBg-py3.9/lib/python3.9/site-packages/vtkmodules/libvtkRenderingTk-9.1.so
/home/nicoco/.cache/pypoetry/virtualenvs/xxx-research-r1WBkZBg-py3.9/lib/python3.9/site-packages/vtkmodules/libvtkRenderingTk.so

Debian bullseye.

Not sure if this is related, but I did not use make install or anything requiring root privileges when building VTK, to try and keep track of what is actually necessary to get my small GUI working. I used python3 setup.py bdist_wheel after make -j and installed the wheel in my poetry virtualenv, through pip.

Anyway, thanks for your help!

dgobbi · February 2, 2022, 3:37pm

While that might work, it’s the wrong fix. But it gave me a clue about where the real fix needs to go, in vtkLoadPythonTkWidgets.py at line 40:

     # create an extensive list of paths to search
-    pathlist = sys.path
+    pathlist = [os.path.join(p, 'vtkmodules') for p in sys.path]

Can you try adding this change to your vtkLoadPythonTkWidgets.py and check to see if it fixes the problem?

nicoco · February 2, 2022, 3:47pm

Yes, it’s a dirty fix.

It does fix the problem! But patching stuff directly in installed packages feels even more dirty

Now tell me all of this will make it to the next official VTK wheel build so I don’t have to bother compiling anything and just use vtk = "^9.1.1" in my pyproject.toml

dgobbi · February 2, 2022, 4:12pm

I will patch vtkLoadPythonTkWidgets.py for the next release (VTK 9.2).

If you want the Tk widget in the official wheels, then consider submitting a VTK patch that adds it to the wheels (see the cmake files here). Otherwise, it just ain’t gonna happen.

nicoco · February 2, 2022, 4:18pm

OK I can try this, thanks for the advice!

nicoco · February 17, 2022, 1:59pm

Hey @dgobbi,

Besides the patch in vtkLoadPythonTkWidgets.py, I now have to manually symlink libvtkRenderingTk-9.1.so to libvtkRenderingTk.so in order to get the tk widget working.

I am unsure if I changed some CMake that results in this behaviour…

Anyway, I have a linux-only dirty fix:

from pathlib import Path
from warnings import warn

import vtk

major, minor, _ = vtk.__version__.split(".")

regular_file = Path(vtk.__file__).parent / f"libvtkRenderingTk.so"
if not regular_file.exists():
    warn("VTK does not seem to have TK support")
else:
    symlink = Path(vtk.__file__).parent / f"libvtkRenderingTk-{major}.{minor}.so"
    if not symlink.exists():
        warn(f"{symlink} does not exists")
        symlink.symlink_to(regular_file)

dgobbi · February 17, 2022, 3:10pm

Do the other VTK libraries have a 9.1 suffix, for example, libvtkCommonCore-9.1.so?

If the cmake configuration changed in some way that removed the version suffix, then that might be the cause of the problem, because vtkLoadPythonTkWidgets.py requires the suffix.

ben.boeckel · February 17, 2022, 4:18pm

There’s the VTK_VERSIONED_INSTALL which indicates whether it has a suffix (and VTK_CUSTOM_LIBRARY_SUFFIX for the actual suffix of libraries; needs a - separator if it is non-empty).

nicoco · March 1, 2022, 6:18am

They don’t, but it doesn’t seem to be a problem. I might be responsible, since CMake is basically black magic to me. Next time I need to build my VTK wheel I’ll try to understand, for now my dirty fix works for me.

As usual, thanks for your answers!