Trouble in pip installing vtk-osmesa

I’ve been using vtk-osmesa installed with pip as pip install --extra-index-url https://wheels.vtk.org vtk-osmesa,
following this post: Status Update: VTK Python Wheels - Announcements - VTK

OS: “Rocky Linux 8.6 (Green Obsidian)”
$ python --version is Python 3.12.4

This times, pip install fails:

$ pip install --extra-index-url https://wheels.vtk.org vtk-osmesa
Looking in indexes: https://pypi.org/simple, https://wheels.vtk.org
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1000)'))': /vtk-osmesa/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1000)'))': /vtk-osmesa/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1000)'))': /vtk-osmesa/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1000)'))': /vtk-osmesa/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1000)'))': /vtk-osmesa/
Could not fetch URL https://wheels.vtk.org/vtk-osmesa/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='wheels.vtk.org', port=443): Max retries exceeded with url: /vtk-osmesa/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1000)'))) - skipping
ERROR: Could not find a version that satisfies the requirement vtk-osmesa (from versions: none)
ERROR: No matching distribution found for vtk-osmesa

I grabbed .whl from https://gitlab.kitware.com/vtk/vtk/-/packages/483 and my problem was fixed.
But I’m happy if simple pip install comes back.

Thanks in advance.

Natsu

Hello @natsu_mizu

That is outdated. Please see latest information at Status Update: Runtime OpenGL render window selection in VTK

There is no longer a need to download a separate package for osmesa support because it is built into the vtk package.

1 Like

That’s great.

I appriceiate that and thank you for such a quick reply.

– Natsu

Hey @jaswantp , I’m seeing similar issues from the wheels and gitlab indexes
```
pip install --extra-index-url https://wheels.vtk.org vtk-osmesa
Looking in indexes: Simple index , https://wheels.vtk.org
ERROR: Could not find a version that satisfies the requirement vtk-osmesa (from versions: none)
ERROR: No matching distribution found for vtk-osmesa
```

I understand that ideally I should not need the separate package, but on my install the fallback to osmesa is temperamental. This is through pyvista.

I was able to manually find the last stable wheel and download it myself, but the indexes seem to not report anything being available.

Dear @jaswantp ,
I understand that nowadays the “vtk” wheel should be used directly, however I am not entirely sure how to enforce fallback to osmesa (or egl) for instance. My tests run out of a custom docker container, which then generates: `ImportError: libXrender.so.1: cannot open shared object file: No such file or directory`

I am not sure what I am supposed to do based on Runtime settings - VTK documentation and Simplified python wheel for VTK - Development - VTK

Any suggestions how to fix this? Is there any python option I can manipulate?

force osmesa or egl by setting the env var VTK_DEFAULT_OPENGL_WINDOW to either vtkEGLRenderWindow or vtkOSOpenGLRenderWindow.

Alternatively, use the specific version of the render window you want to use.

I have not tried those environment variables yet, but I was specifying that I wanted off_screen rendering in my pyvista calls. It could just be an issue with pyvista.

For now though, I’m still interested in being able to query the wheels index so I don’t break previously committed environments. Can we get confirmation if these are coming back or not?

They are not coming back.

It could just be an issue with pyvista

Probably

I think there are two issues:

The vtk-osmesa for release 9.3.1, the last release of that package, was deleted from wheels.vtk.org a few days ago. So previous references on how to install this wheel is lo longer valid. The package is still available through VTK Gitlab, though, for anyone that needs it (please do not delete it from there as well).

Second issue is probably that while the vtk-osmesa package was built without X libraries, the releases 9.4 and 9.5 had no packages without X dependencies. This changes in 9.6, when also the X libraries are dynamically loaded, and maybe it’s a good idea to try the latest nightly build of VTK and see if that solves the problems?

I have no problems setting up the nightly builds of VTK with OSMesa in an environment without the X libraries, where I previously depended on the vtk-osmesa. I think for many pyvista workflows this is the way forward.

Is that true? Looking at wheels.vtk.org, I see

and others.

$ python3 -m venv vtk-osmesa-venv
$ . vtk-osmesa-venv/bin/activate
$ pip install --extra-index-url https://wheels.vtk.org vtk-osmesa
Looking in indexes: https://pypi.org/simple, https://wheels.vtk.org
ERROR: Could not find a version that satisfies the requirement vtk-osmesa (from versions: none)
ERROR: No matching distribution found for vtk-osmesa

On Friday everything worked, Saturday this started happening. This is testing with Python 3.12.

And yes, through Gitlab the wheels can still be downloaded, but wheels.vtk.org no longer appear to serve the vtk-osmesa 9.3.1 wheel (as demonstrated above).

@cory.quammen since the packages are there (and they are, I’m able to download them) but pip is unable to retrieve them (it isn’t able on my system, at least), then is it possible that vtk.org recently went through some security-related or protocol-related changes that would cause pip to fail like this?

Edit: example using pip to get vtk from kitware’s servers vs. from pypi:

pip --verbose index --pre --index-url https://wheels.vtk.org versions vtk
ERROR: No matching distribution found for vtk
pip --verbose index --pre versions vtk
vtk (9.5.2)
Available versions: 9.5.2, 9.5.1, 9.5.0, 9.5.0rc3, 9.5.0rc2, 9.5.0rc1, 9.4.2, 9.4.1, 9.4.0, 9.4.0rc3, 9.3.20230807rc0, 9.3.1, 9.3.0, 9.3.0rc2, 9.3.0rc1, 9.2.6, 9.2.5, 9.2.4, 9.2.2

Edit 2: apparently pip needs more verbosity to show why it fails:

pip -vv index --index-url https://wheels.vtk.org versions vtk

The result is a lot of text that I won’t paste here, but it looks like kitware’s servers think that the pip requests are coming from a bot and are therefore blocking them.

Hi @dgobbi,

On Friday, Kitware enabled Anubis to filter out traffic from AI scrapers that was making gitlab.kitware.com unusable. Since the wheels listed on wheels.vtk.org download from gitlab.kitware.com, that protection is almost definitely interfering with the pip downloads.

I will work with our system administrators to remove the block.

@cory.quammen was @mwestphal mistaken then? I directly asked if the wheels.vtk.org and gitlab indexes were coming back and he said no.

I think he meant they aren’t coming back in the sense that they will not be provided in future versions of VTK.

1 Like

Just to clarify, dedicated osmesa or egl wheel won’t be created for any futures version of VTK as the current “vtk” wheel already include them using runtime selection via glad.

I was indeed mistaken as I misunderstood the issue completely.

Indeed, no new vtk-osmesa wheel will be provided in the future because the single wheel supports it since vtk 9.5.0, but currently available wheel of older release, osmesa or otherwise should stay available and I hope this issue will be fixed soon.

Hey @cory.quammen ,

If I understood your reply. The wheels (9.3.1 in that case) are not being deleted but it’s mostly a temporary issue with Anubis filter and it will soon be fixed.

Right ? :grimacing:

Correct :slight_smile:

The issue with Anubis should be fixed now. Please try pip installing again.

1 Like