One option is to use VTK OpenVR binaries bundled with Paraview (for engineering applications) or 3D Slicer (for medical applications).
OpenVR classes are not Python-wrapped (at least in 3D Slicer) but interface classes are added that are Python-wrapped, which can be used to set up the virtual reality view (and they also provide continuous rendering while you run Python scripts or interaction the Python terminal). You can use Slicer’s Python environment as any other Python environment (you can pip install any packages, etc.). If you have any questions about using or Python scripting in Slicer, you can post it to https://discourse.slicer.org.