I made this post to help ImGui/3rd party UI users who wish to display VTK content in an externally provided context(glad/glbindings/…) and (or) windowing system (glfw/…) or DearImGUI, Nuklear, etc.
So far, there is only one post which discusses procedures to render VTK into user provided frame buffer. It is quite outdated and does not function for VTK >= 9.0, probably since much of VTK’s OpenGL code base has changed.
I’ve updated a fork of the repo to conform with imgui_impl_xxx.h files and vtkGenericOpenGLRenderWindow. It is a straight forward usage for DearImGUI users, but not the case for VTK users.
vtkImGuiAdapter provides two classes: vtkImGuiSDL2OpenGLRenderWindow (subclass of the vtkSDL2OpenGLRendeWindow available since 9.0) and vtkImGuiSDL2RenderWindowInteractor.
vtkImGuiAdapter takes the path to override Frame and let ImGui loop control the re-rendering. The render window is owned by VTK, and ImGui writes on it. I am sure optimizations can be done by someone with more OpenGL knowledge than myself to avoid some unnecessary re-renderings.
It has worked for me pretty well to put up some demos really quick.
I noticed your post when I first searched vtk + imgui on here and sort of borrowed some parts of your SDL2RenderWindow to implement vtkGlfwRenderWindow
The render window is owned by VTK, and ImGui writes on it.
Right. My implementation does the opposite thing. The window manager(provided by the user!) owns the windows and VTK renders its stuff onto a DearImGUI window.
The render window is generic, think of it as a dummy window.
The render window interactor is also generic, defaults to platform-native interactors. (vtkXRenderWindowInteractor on Linux, vtkWin32OpenGLRenderWindowInteractor on win, cocoa on mac)
The opengl context is user-provided.
This would be suitable for folks who’d like to embed a VTK visualization into an existing DearImGUI app as opposed to embedding DearImGUI windows into a VTK visualization/app.
I originally had to set this up for prototyping purposes. Now, I moved on to using ParaView for everything so I figured it’d be a good idea to push and update that out-dated imgui-vtk repo.
This specific style of embedding OpenGL scenes into DearImGUI is done in almost all indie game-engines. (see Dear ImGUI render to viewport).
Hi @jaswantp, I have failed to run your fork using VTK version: 9.0.20210114. Basically nothing gets rendered inside the VTK sub-window. On the bright side, there are no crashes.
Have you played with it recently? Should I try another VTK version?
I’m surprised you did not see a spam of such OpenGL errors. Perhaps you built VTK to not display OpenGL errors.
If you turned on VTK_REPORT_OPENGL_ERRORS and VTK_REPORT_OPENGL_ERRORS_IN_RELEASE_BUILDS, your console would be spammed with the errors in the image above.
Anyway, it was displaying a blank window because of a stupid typo where I used GL_DRAW_BUFFER instead of GL_DRAW_FRAMEBUFFER in the bind and unbind calls around Render(). This is the invalid enum that OpenGL complains about.
With this fix commit, it should work! Let me know if it doesn’t.
There are three things you could do to get it working.
Retry from my repo.
vtk 9.0.1
vtk 8.2.0. Refer to upstream? It has a fix for backward compatibility up to 8.2.0 that binds the application fbo (g_FBOHdl) to GL_FRAMEBUFFER instead of GL_DRAW_FRAMEBUFFER.
Thanks for bringing this up. I’ll have to keep the source up to date with the constant updates being made to the vtkOpenGLRenderWindow::Frame() since 9.0.0
I’m also interested in your build setup. Did you link with ParaView’s VTK? Or is it bare VTK? Do you use the demo provided context loader i.e, glad? Or does it default to system identified OpenGL loader like GLEW/etc…
I didn’t know about VTK_REPORT_OPENGL_ERRORS , really useful .
Right :), it’s the only way to find errors in OpenGL with vtk.
I used bare VTK 9, with SDL2, and GLEW as loader.
Good to know that DearImGUI from demo app was able to render itself into a context provided by GLEW.
By the way, the VTK_BUILD_VERSION in development is a date (i.e 20210114 ), I suspect #if VTK_BUILD_VERSION >= 2 isn’t filtering anything.
I initially used it as a hack to separate logic for anything <= 9.0.1. This commit, post v9.0.1 introduces a render frame buffer. From what I understood in the source, it required changing the enum argument of glBindFrameBuffer in my code.
So, GL_FRAMEBUFFER(for <= 9.0.1) → GL_DRAW_FRAMEBUFFER (for anything > 9.0.1)).
This will let VTK blit its render frame buffer into our frame buffer which will be rendered onto a texture eventually displayed inside a DearImGUI window.
@phcerdan , if you’re interested, do you mind giving this a spin with your VTK build?
Note: Clone the imgui submodule too.
I made a rapid prototype that intercepts interactive events from VTK and injects the overlay right before Frame is called.
The important change is that
requires no 3rd party windowing system!
No need to render to texture! It uses the same render buffer that VTK will use as a read-buffer and blit into display buffer. This blit occurs in Frame call.
For me it works on windows with VTK master branch.
If you see the test_imgui_vtk.cxx file, injection is primarily done with one line.
I haven’t yet tested it on Linux though. So yes, there might be some bugs there. I’m developing it on Windows using VS2019 community. I prefer this IDE for its simple and usable debugger.
I wish to develop this into a state explorer for VTK based apps. Essentially, traverse the vtk pipeline from renderer > actors > mappers > algorithms/sources > inputs.
I’d imagine ImGui tree widgets/tables that expand to show the properties available with vtkGet/vtkSet macros in real time and possibly small textures of the output before and after a filter is applied. For the renderer and all actors, I want to display and set the transform matrices, possibly with sliders. I hope this would improve my understanding of matrix transformations.
On windows, one could technically inject a line of code dearImGUIinjector->Inject(interactor) into paraview.exe process and use the interactor object of ParaView’s active RenderView, possibly a QVTKInteractor. Since ParaView is open source, it makes it a little bit easier to do dll injection.
I don’t know how long it’s gonna take, but I’d like to see this happen eventually.
This is by design. ImGUI stops forwarding mouse events to VTK when the mouse is inside imgui window. But I think this is not expected of an overlay! Thanks for mentioning that! I will add a checkbox to grab mouse input.
About the imgui assertion error, I’ll debug it on a linux box and see what I can do… maybe it has to do with the implementation of chrono::system_clock on nix vs windows c++ libraries.
Alright, I debugged it over the weekend and my assumptions were partly right.
Bug: On nix platform, Render function was called inconsistently, so io.DeltaTime was <= 0ms
Cause: Since VTK doesn’t call Render in every iteration of the event loop, I had to create a one-shot timer for 0ms so that the event queue will always have a TimerEvent that will call my render function. This is to make sure the overlay is drawn on every frame. Apparently, the x11 Render window interactor doesn’t behave well with 0ms timers, as a matter of fact, it doesn’t even fire the timer.
I’ll have to delegate vtkDearImGuiInjector to call ProcessEvents effectively taking over the event loop. This could be achieved if interactor framework exposed HandleEventLoop via vtkBooleanMacro. I want to do this without deriving vtkRenderWindowInteractor for simplicity at the injection site. Since it’s default value is false, its okay for now!
I spent a good weekend debugging the Emscripten side of things in VTK. In the end, I got this to compile with Emscripten for WebAssembly. The simple cone example with DearImGUI. This uses GLES 3.0, some VTK functionality will not work…
I am using my c++14 project in vs2019 to run vtkImGuiAdapter-project, there is #include <GLES3/gl3.h> specified in example-imgui-vtk-sdl
In order to use gl3.h, I have downloaded windows 64bit gles3.0.4 and 3.0.2 and 32bit 3.0.2 from OpenGL ES Emulator – Arm Developer and make setting in the example-imgui-vtk-sdl to use the headers, lib, and dll files. However when I ALL_BUILD ->build the project, there is this same error in libEGL.dll for all gles downloaded, the error is
