vtk+DearImGui integration through vtkGenericOpenGLRenderWindow

Development
#1

I made this post to help ImGui/3rd party UI users who wish to display VTK content in an externally provided context(glad/glbindings/…) and (or) windowing system (glfw/…) or DearImGUI, Nuklear, etc.

So far, there is only one post which discusses procedures to render VTK into user provided frame buffer. It is quite outdated and does not function for VTK >= 9.0, probably since much of VTK’s OpenGL code base has changed.

I’ve updated a fork of the repo to conform with imgui_impl_xxx.h files and vtkGenericOpenGLRenderWindow. It is a straight forward usage for DearImGUI users, but not the case for VTK users.

Alternative:
It would be easier to have a class vtkDearImGuiVport; sub-class vtkGenericOpenGLRenderWindow. In an override to vtkGenericOpenGLRenderWindow::Frame(), access texture id from vtkOpenGLRenderWindow::DrawPixelsTextureObject and pipe it to ImGui::Image(…). This would remove the additional render FBO to texture done in the current repo. But a huge problem arises when StereoRender is off since the internal texture object is not filled in vtkRenderWindow::StereoRenderComplete().

Maybe in a default path inside vtkRenderWindow::StereoRenderComplete(), call vtkOpenGLRenderWindow::GetPixelData(..) to fill in vtkOpenGLRenderWindow::ResultFrame and break away. This will ensure vtkOpenGLRenderWindow::ResultFrame is populated. From then on, vtkOpenGLRenderWindow::DrawPixels will initialize vtkOpenGLRenderWindow::DrawPixelsTextureObject.

vtkImGuiDemo

DearImGUI Gallery thread
Edit: update links
Edit: Add a link to DearImGui gallery post

1 Like
#2

Thanks for sharing @jaswantp!

I had a go on integrating vtk+ImGui: GitHub - phcerdan/vtkImGuiAdapter: Adapter to add imgui to an existing vtkRenderWindow using SDL2.

vtkImGuiAdapter provides two classes: vtkImGuiSDL2OpenGLRenderWindow (subclass of the vtkSDL2OpenGLRendeWindow available since 9.0) and vtkImGuiSDL2RenderWindowInteractor.

vtkImGuiAdapter takes the path to override Frame and let ImGui loop control the re-rendering. The render window is owned by VTK, and ImGui writes on it. I am sure optimizations can be done by someone with more OpenGL knowledge than myself to avoid some unnecessary re-renderings.

It has worked for me pretty well to put up some demos really quick.

vtkImguiAdapter_example
vtkImguiAdapter_example1038×996 1000 KB

Here is the code example that generates that gif.

Link to the discourse post where I first introduced it

#3

Thanks!

I noticed your post when I first searched vtk + imgui on here and sort of borrowed some parts of your SDL2RenderWindow to implement vtkGlfwRenderWindow :slight_smile:

The render window is owned by VTK, and ImGui writes on it.

Right. My implementation does the opposite thing. The window manager(provided by the user!) owns the windows and VTK renders its stuff onto a DearImGUI window.

  • The render window is generic, think of it as a dummy window.
  • The render window interactor is also generic, defaults to platform-native interactors. (vtkXRenderWindowInteractor on Linux, vtkWin32OpenGLRenderWindowInteractor on win, cocoa on mac)
  • The opengl context is user-provided.

This would be suitable for folks who’d like to embed a VTK visualization into an existing DearImGUI app as opposed to embedding DearImGUI windows into a VTK visualization/app.

I originally had to set this up for prototyping purposes. Now, I moved on to using ParaView for everything so I figured it’d be a good idea to push and update that out-dated imgui-vtk repo.

This specific style of embedding OpenGL scenes into DearImGUI is done in almost all indie game-engines. (see Dear ImGUI render to viewport).

1 Like
#4

Hi @jaswantp, I have failed to run your fork using VTK version: 9.0.20210114. Basically nothing gets rendered inside the VTK sub-window. On the bright side, there are no crashes.

Have you played with it recently? Should I try another VTK version?

#5

Hello, This happens when build version >= 2. This is my output for ParaView 5.9.0’s VTK which is 9.0.20201030

image
image1905×512 48.8 KB

I’m surprised you did not see a spam of such OpenGL errors. Perhaps you built VTK to not display OpenGL errors.

If you turned on VTK_REPORT_OPENGL_ERRORS and VTK_REPORT_OPENGL_ERRORS_IN_RELEASE_BUILDS, your console would be spammed with the errors in the image above.

Anyway, it was displaying a blank window because of a stupid typo where I used GL_DRAW_BUFFER instead of GL_DRAW_FRAMEBUFFER in the bind and unbind calls around Render(). This is the invalid enum that OpenGL complains about.

With this fix commit, it should work! :slight_smile: Let me know if it doesn’t.

There are three things you could do to get it working.

  1. Retry from my repo.
  2. vtk 9.0.1
  3. vtk 8.2.0. Refer to upstream? It has a fix for backward compatibility up to 8.2.0 that binds the application fbo (g_FBOHdl) to GL_FRAMEBUFFER instead of GL_DRAW_FRAMEBUFFER.

Thanks for bringing this up. I’ll have to keep the source up to date with the constant updates being made to the vtkOpenGLRenderWindow::Frame() since 9.0.0

I’m also interested in your build setup. Did you link with ParaView’s VTK? Or is it bare VTK? Do you use the demo provided context loader i.e, glad? Or does it default to system identified OpenGL loader like GLEW/etc…

1 Like
#6

That fixes it, thanks! :partying_face:
I didn’t know about VTK_REPORT_OPENGL_ERRORS, really useful :slight_smile:.

Do you use the demo provided context loader i.e, glad? Or does it default to system identified OpenGL loader like GLEW/etc

I used bare VTK 9, with SDL2, and GLEW as loader.

By the way, the VTK_BUILD_VERSION in development is a date (i.e 20210114), I suspect
#if VTK_BUILD_VERSION >= 2 isn’t filtering anything.

#7

Hi @phcerdan

I didn’t know about VTK_REPORT_OPENGL_ERRORS , really useful :slight_smile:.

Right :), it’s the only way to find errors in OpenGL with vtk.

I used bare VTK 9, with SDL2, and GLEW as loader.

Good to know that DearImGUI from demo app was able to render itself into a context provided by GLEW.

By the way, the VTK_BUILD_VERSION in development is a date (i.e 20210114 ), I suspect
#if VTK_BUILD_VERSION >= 2 isn’t filtering anything.

I initially used it as a hack to separate logic for anything <= 9.0.1. This commit, post v9.0.1 introduces a render frame buffer. From what I understood in the source, it required changing the enum argument of glBindFrameBuffer in my code.

So, GL_FRAMEBUFFER(for <= 9.0.1) → GL_DRAW_FRAMEBUFFER (for anything > 9.0.1)).

This will let VTK blit its render frame buffer into our frame buffer which will be rendered onto a texture eventually displayed inside a DearImGUI window.

1 Like