Hi all, just today I merged a patch into VTK-master to improve the vtk-python method descriptors and docstrings. This should help autocompletion and hinting for ipython and jupyter. Anyone who wants to try it out, please do so, because VTK 9.1 is coming soon and I won’t have time to do additional testing myself.
if you have an IDE that uses .pyi files, there is a script that you can run to generate them. Note that this script will only work with the current master branch, it will not work with yesterday’s master or with VTK 9.0.3.
@amaclean since you’re a PyCharm user, is there a chance that you can try this script to see if it makes any difference to the user experience? When run with no arguments, the script will simply deposit the .pyi files next to the .pyd files.
Used Linux and it seems to work nicely. I ran it with no parameters. One suggestion, it might be good to mention that if the environment is not set up properly call it like this:
vtkpython ./vtk_generate_pyi.py
As regards PyCharm, it doesn seem to make any difference. PyCharm seems to do this when it loads the modules. However PyCharm is aware of the files because when I run vtk_generate_pyi.py it reindexes and also when these files are deleted.
Thanks for giving it a spin, Andrew. If the .pyi files cause PyCharm to reindex, then at least I know that it’s pulling information from them. And some of that information (e.g. @overload hints) is only available if the .pyi files are present.
@phcerdan you mentioned issues with vtk-python autocompletion in an earlier thread, If you can try the current master (or vtk-9.1-rc1 when it comes out in a few days), any feedback you can give would be greatly appreciated.
Fast check: I am using pyright (language server) in vim trough the plugin coc.vim.
I installed vtk in a python virtual environment from a wheel generated from the build tree of current master. And then I used the script to generate the stub pyi files.
I am able to get the info from .pyi stubs files in the editor. However I have to specify
the vtkmodules path.
If I import the whole vtk package import vtk I can only access inner python methods
However if I import using the vtkmodules path it works flawlessly.
Thanks, Andrew. One of the features I’m hoping people will find useful is the type hints, particularly for overloaded methods:
This is the feature that I’m most interested in people testing on the various IDEs and Python consoles.
I strongly suspect that the ‘vtkCameraOrientationWidget’ error is due to leftovers on your system from an old build or something like that. The fact that it mentions an __init__.py makes me think your system might have a vtkInteractionWidgets folder somewhere that it doesn’t belong.
I do not have any immediate plans to add the .pyi files to the VTK build, but wouldn’t object if someone else did so. I’ve found that ipython, jupyter, and the default Python console actually do fairly well just with the updated docstrings in the wrappers, and for them the .pyi files don’t seem to add much benefit (though I would be interested to hear from others on the subject).
@phcerdan thanks for giving it a spin. Were you able to get any hinting for method argument types?
The script only generates vtkmodules .pyi files, it would need to be modified to generate a vtk.pyi (and the file would be very large). It’s something I could do, but I’m not convinced that it would be a good idea.
Thanks a lot @dgobbi! This is awesome already. If at some point this could be available directly after pip installing vtk it would be great. /PyCharm user
For those of you using import vtk you can easily start using the vtkmodules files. I have a script that looks through your code and provides you with a list of the necessary from ... import ... statements.
Please see VTKImportsForPython, and ignore the comment about sorting. It needs testing as I am not sure that I capture all of the VTK classes since pattern matching is used.
Also there is a script VTKModulesForCxx that will generate a find_package(VTK COMPONENTS ...) command for CMake, useful for Cxx development.
Both these scripts have some minor changes coming through (tomorrow?).
I would prefer to add the stubs directly to the vtk package. But as an alternative, another optional package vtk-stubs, vtk-pyi or similar name, can be created. The only inconvenient is that vtk and vtk-stubs might get out of sync with each other when updating vtk but not vtk-stubs.
The setup.py.in file seems the place to trigger the stubs script after the build? I am not familiar with vtk python packaging.
My inclination would be to trigger the script with cmake during the build, rather than with setup.py after the build. Putting the .pyi files in the build tree would allow us to add a test for them (e.g. the test could load them into Python to check the syntax, and could check for unresolved types, etc). But I don’t think we’re quite there yet.
Also, for people who are using “import vtk”, you might have better luck with this instead:
import vtkmodules.all as vtk
That should be sufficient for PyCharm to take the “vtk” hints from “vtkmodules”. I don’t know whether it will work for other IDEs, however, since every IDE does hints differently.
Thanks @dgobbi, it works really well in PyCharm, as soon as you type vtk. you get a dropdown list of all the classes etc. and the modules they live in! It’s really fast too. Love it, thanks for the tip!
I’ve been crazy busy and haven’t followed what’s going on with Python, sorry about that.
This morning I did a clean master pull / VTK build to test a new filter. I found that the way I’ve been testing/debugging Python test scripts for the last many years no longer works. For example:
Hi Will, as long as your version of Python is somewhere between 3.6 and 3.9, running vtkpython as you have indicated should still work just fine. That’s how I usually run my test scripts, too. Most of the vtk-python tests haven’t changed in 5 years or more, so backwards compatibility should be pretty good.
Traceback (most recent call last):
File “TestExtractPoints.py”, line 2, in
import vtk
File “/home/will/Develop/Bisect/release/lib/python3.6/site-packages/vtk.py”, line 31, in
all_m = importlib.import_module(‘vtkmodules.all’)
File “/usr/local/lib/python3.6/importlib/init.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “/home/will/Develop/Bisect/release/lib/python3.6/site-packages/vtkmodules/all.py”, line 7, in
from .vtkCommonCore import *
ImportError: /home/will/Develop/Bisect/release/lib/python3.6/site-packages/vtkmodules/vtkCommonCore.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PyExc_IndexError
This error means that libpython (libpython3.6.so) is not being linked, or perhaps there is a symbol visibility problem. @ben.boeckel do you have ideas about what might cause this?
