QVTKRenderWindowInteractor and QWebEngineView conflict

[sapvi]

I want to use QVTKRenderWindowInteractor for displaying some 3D data and, in the same app but in tab viewer, I want to have QWebEngineView to render interactive bokeh plots, i.e. each visualization has its own tab.
It seems not possible, however. Both widgets work ok when separated, but QWebEngineView is not rendered once I put both to the same app where I have QVTKRenderWindowInteractor.
I tried:

1. First creating QWebEngineView and then QVTKRenderWindowInteractor and vice versa.
2. I tried to use the following lines in case the problem is that both are trying to use GPU.

os.environ["QTWEBENGINE_DISABLE_GPU"] = "1"
os.environ["QTWEBENGINE_CHROMIUM_FLAGS"] = "--disable-gpu --disable-software-rasterizer"

I have seen this workaround - Cannot use QVTKRenderWindowInteractor and QWebEngineView together in one application.. But I would like to find some easier workaround if possible.

Questions:

1. Is the problem that they are fighting for GPU or rather for some other resource?
2. Can there be some other workaround instead of embedding code?
3. Can I use something else other than QWebEngineView that would not create conflict but would allow me to create interactive plots? I have thought about rendering png and just setting it as image to some simpler widgets, but sadly it is not possible as I really need user to have possibility to interactively navigate in the plot.
4. Does it help that the visualizations are in different tabs, i.e. user does not see them simultaneously? Can I somehow “stop” one visualization when closing the corresponding tab, but without long waiting time for switching tabs?
5. Any other alternative to bokeh that does not require QWebEngineView to render and thus does not create conflict?

Problem in MRE below. I am on Windows 11, Python 3.11. I need to render VTK on GPU but I am ok with non-GPU rendering for QWebEngineView.

import sys
import xyzservices.providers as xyz
from PySide6.QtWidgets import QApplication, QMainWindow, QVBoxLayout, QWidget, QTabWidget
from PySide6.QtWebEngineWidgets import QWebEngineView
from bokeh.plotting import figure
from bokeh.resources import CDN
from bokeh.embed import file_html
from vtkmodules.qt.QVTKRenderWindowInteractor import QVTKRenderWindowInteractor
import vtk


class MapViewer(QWidget):
    def __init__(self):
        super().__init__()

        tile_provider = xyz.OpenStreetMap.Mapnik
        self.plot = figure(
            x_range=(-2000000, 6000000),
            y_range=(-2000000, 6000000),
            x_axis_type="mercator",
            y_axis_type="mercator",
            sizing_mode="stretch_both",
        )
        self.plot.add_tile(tile_provider)

        # Convert Bokeh plot to HTML
        self.web_view = QWebEngineView()
        html = file_html(self.plot, CDN, "Bokeh Map")
        self.web_view.setHtml(html) 

        layout = QVBoxLayout()
        layout.addWidget(self.web_view)
        self.setLayout(layout)


class VTKViewer(QWidget):
    def __init__(self):
        super().__init__()

        self.vtk_widget = QVTKRenderWindowInteractor(self)
        layout = QVBoxLayout()
        layout.addWidget(self.vtk_widget)
        self.setLayout(layout)

        # Setup VTK Renderer
        self.renderer = vtk.vtkRenderer()
        self.vtk_widget.GetRenderWindow().AddRenderer(self.renderer)
        self.iren = self.vtk_widget.GetRenderWindow().GetInteractor()

        # Create a random 3D VTK Object (Sphere)
        sphere = vtk.vtkSphereSource()
        sphere.SetRadius(5.0)

        mapper = vtk.vtkPolyDataMapper()
        mapper.SetInputConnection(sphere.GetOutputPort())

        actor = vtk.vtkActor()
        actor.SetMapper(mapper)
        actor.GetProperty().SetColor(0, 1, 0) 

        self.renderer.AddActor(actor)
        self.renderer.SetBackground(0.1, 0.1, 0.1)  

        self.iren.Initialize()


class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()
        self.setWindowTitle("Tab Widget with Bokeh Map and VTK 3D")
        self.tabs = QTabWidget()

        # Tab 1: Bokeh Map Viewer
        self.map_viewer = MapViewer()

        # Tab 2: VTK Viewer
        self.vtk_viewer = VTKViewer()

        self.tabs.addTab(self.map_viewer, "Map Viewer")
        self.tabs.addTab(self.vtk_viewer, "VTK 3D Viewer")

        self.setCentralWidget(self.tabs)
        self.resize(1200, 800) 


if __name__ == "__main__":
    app = QApplication(sys.argv)

    main_win = MainWindow()
    main_win.show()

    sys.exit(app.exec())

[dgobbi (David Gobbi)]

The point of conflict is that QVTKRenderWindowInteractor uses a native vtkRenderWindow. What I mean by “native” is that it uses vtkWin32OpenGLRenderWindow on Windows, vtkXOpenGLRenderWindow on linux, etc. These native windows create their own OpenGL context, and this conflicts with some Qt classes.

To make things work, it’s necessary for Qt to do the GPU setup and create the OpenGL context, and then VTK can draw in the context that Qt created. This is impossible with the native vtkRenderWindow classes, so many years ago someone created the vtkGenericOpenGLRenderWindow class specifically for this purpose. But QVTKRenderWindowInteractor is a much older class and it doesn’t use vtkGenericOpenGLRenderWindow. Someone started a project to make these classes work together once, but never finished. I’ll try to find that project and link to it here.

So, until someone makes a Python QVTKRenderWindowInteractor class that uses vtkGenericOpenGLRenderWindow, I don’t think it’s possible to ensure perfect compatibility between VTK and Qt in a pure-python (versus C++/embedded) application.

[sapvi]

Thank you! It seems now I at least understand why this happens.

Do you know if there is actually way to use vtkGenericOpenGLRenderWindow in Python? I guess it would only know how to render the (in my case) point cloud, I’d need to implement any basic navigation to change view point (I do not need any other form of interaction)? Or it is more complicated than that?

[dgobbi (David Gobbi)]

The vtkGenericOpenGLRenderWindow is just a vtkRenderWindow that expects something else to provide the OpenGL context. So questions about interaction don’t really apply, since interaction isn’t what a vtkRenderWindow is responsible for. Interaction is either done via a vtkRenderWindowInteractor (or via Qt events), just like most VTK programs.

As far as I know, there aren’t any working examples of vtkGenericOpenGLRenderWindow in Python. The best example is a C++ example: the QVTKRenderWindowAdapter class that comes with the VTK source code. If you want to read through that example, also take note of the documentation to understand how vtkGenericOpenGLRenderWindow is to be used:

vtkGenericOpenGLRenderWindow provides a skeleton for implementing a render window using one’s own OpenGL context and drawable. To be effective, one must register an observer for WindowMakeCurrentEvent, WindowIsCurrentEvent and WindowFrameEvent. When this class sends a WindowIsCurrentEvent, the call data is a bool* which one can use to return whether the context is current.

There is also an unfinished port of QVTKRenderWindowAdapter from C++ to Python, which you can find here: merge request !9443