Building and publishing Nightly Python Wheel

Support
1

Are there any plans to implement Python 3.10/3.9 nightly wheels based on the latest VTK master?

2

The wheels are already built nightly during testing, for Python 3.6 through 3.10. They’re just hard to find.

Go to the scheduled pipelines, select the last nightly pipeline, and from there you can find the wheel jobs. Then download the build artifacts to get the wheel. For example, yesterday’s Windows wheel for Python 3.9 (link will only be valid for one day).

The hard part is generating persistent links. I don’t know how that would be done.

3

The hard part is generating persistent links. I don’t know how that would be done.

Uploading the wheels as GitHub release artifact associated with the project kitware/VTK and a tag called nightly-python-wheels would allow to have a stable link.

Such link would also allow pip installing using the find-links feature of pip.

To avoid bumping this tag daily, the associated GitHub VTK release could be associated with an orphan branch also called nightly-python-wheels including a README.md.

To avoid the number of assets to grow indefinitely, a scheduled job would clear up the assets older than N days.

And the tool for uploading assets would be this one: GitHub - github-release/github-release: Commandline app to create and edit releases on Github (and upload artifacts)

4

You would only need the nightly and possibly the previous nightly.

This has just arisen for me because at present I cannot build python wrapping properly in Windows. So I thought that this might be an alternative.

In Linux, wrapping is Ok using Python 3.10 although vtk-python doesn’t work (can’t find numpy) but using Python is OK.

5

4 posts were split to a new topic: Setting PATH environment variables for VTK build tree on Windows

6

We had them going to PyPI, but we maxed out our quota there pretty fast.

AFAIK, there are no quotas on GitHub for releases. If not why clean them up? They can help bisect issues down.

There are already CI jobs which upload the wheels to places; this can be another one (see .gitlab/rules.yml for which one runs on the weekly schedule and we’ll need a token to perform the upload).

7

I have expanded on the comments by @dgobbi (Thanks David). These instructions are for getting the Windows Wheels, but are easily modified for other operating systems. I hope they are useful for others.

Kitware Wheels

The wheels are already built nightly during testing, for Python 3.6 through 3.10. They’re just hard to find.

Go to the scheduled pipelines, select the last nightly pipeline and from there you can find the wheel jobs. Then download the build artifacts to get the wheel. To do this click on the link and edit it adding /artifacts/browse/build/dist/ to the end of the link. This will take you to a page allowing you to download the wheel.

Then download it to a folder.

Note:

  • The nightly wheel builds will generally be finished by UTC 06:00

  • These links are only valid for one day and will look something like this: https://gitlab.kitware.com/vtk/vtk/-/jobs/6603303/artifacts/browse/build/dist/, where the the number 6603303 will change daily.

Setup a virtual environment

We are assuming that the latest python release from Python is already installed, Python 3.10 in this case. Also we are working in Windows 10/11.

If you haven’t already done this, set up a new virtual environment e.g. VTK-310-Wheel.


   deactivate

   py -m venv %USERPROFILE%\venv\VTK-310-Wheel

   %USERPROFILE%\venv\VTK-310-Wheel\scripts\activate

   python -m pip install --upgrade pip

   pip install Sphinx numpy matplotlib pyOpenSSL

   pip install wheel

Installing the wheel

Do this whenever you need a new nightly wheel.


   %USERPROFILE%\venv\VTK-310-Wheel\scripts\activate

   python -m pip install --upgrade pip

   pip install <path to the wheel>

e.g. pip install D:/Users/<username>/Downloads/vtk-9.1.20220213.dev0-cp310-cp310-win_amd64.whl

Note:

  • You don’t necessarily have to upgrade pip every time.

Using an IDE

In the case of Pycharm, after you have set up your project:

  • Go to File|Settings|Python Interpreter

  • Click on the wheel symbol, select Add and then Virtualenv Environment

  • Select Existing Environment and find your interpreter. Its path will be %USERPROFILE%\venv\VTK-310-Wheel\Scripts\Python.exe.

  • You’re done!

1 Like
8

Hello All,

I would like to be able to build a VTK Python wheel myself. I have seen that on GitLab there are jobs named for example wheel-windows-python310-x86_64:build I am not familiar with GitLab does anyone have any tip on how to dig into the job files which would serve as an example? Are they available in some form in a git repository?

Thanks for any advise,

Best Regards,

Andrea

9

General guidance is in Documentation/dev/build.md, but specific instructions and configure flags are in .gitlab/os-*.yml and .gitlab/ci/configure_*.cmake.

10

Thank you! I have found the information I needed.

Best Regards,

Andrea

11

Hi! Please can you add “linux arm64” to nightly / weekly builds?

12

It’d be nice, but there’s a lack of resources for it right now. I’d have to check if we had the hardware to do any testing before I’d be comfortable releasing official wheels at least.

13

Thanks! I have compile deb packages for rpi like so. I pushed the the wheel build var though to debuild with --set-envvar=“VTK_WHEEL_BUILD=ON” as it doesn’t look like Debian enable that

https://salsa.debian.org/science-team/vtk9/-/raw/master/debian/rules

apt install -y chrpath cmake debhelper-compat default-jdk default-libmysqlclient-dev dh-sequence-python3 doxygen-latex \
    ghostscript gnuplot-nox graphviz hdf5-tools dh-sequence-javahelper libalglib-dev javahelper libavcodec-dev \
    libavformat-dev libavutil-dev libdigest-md5-perl libdouble-conversion-dev libeigen3-dev libexpat-dev \
    libfontconfig1-dev libfreetype6-dev libftgl-dev libgdal-dev libgl1-mesa-dev libgl-dev libgl2ps-dev \
    libglew-dev libglu1-mesa-dev libglu-dev libhdf5-openmpi-dev libhpdf-dev libjpeg-dev libjs-mathjax \
    libjsoncpp-dev liblz4-dev libmrmpi-dev libnetcdf-cxx-legacy-dev libnetcdf-dev libogg-dev libosmesa6-dev \
    libpng-dev libpq-dev libproj-dev libsqlite3-dev libswscale-dev libtbb-dev libtheora-dev \
    libtiff-dev libutfcpp-dev libx11-dev libxdmf-dev libxml2-dev libxpm-dev libxt-dev lzma mpi-default-dev \
    pkg-config python3-dev python3-matplotlib python3-mpi4py qtbase5-dev libqt5opengl5 \
    r-base-core r-base-dev r-base xauth xvfb

  git clone https://salsa.debian.org/science-team/vtk9.git
  cd vtk9
  dch -a "compiled by djsftree"
  debuild -i -us -uc -b --set-envvar="VTK_WHEEL_BUILD=ON"
  /usr/lib/python3 -m pip install wheel
  /usr/lib/python3 setup.py bdist_wheel
  /usr/lib/python3 -m pip install dist/vtk-*.whl
14

An envvar does nothing to VTK; it needs to be passed to the CMake command that builds VTK. Also, I don’t think debbuild of the stock vtk9 package is likely to accept the layout the wheel build ends up making.