Before, I start, I just want to say that vtk.js is an absolute joy to work with, and appreciate the people @ kitware who have been working on it.
However, right now I am finding vtk.js to not be enough for what I am trying to do (both performance + feature-wise). For instance, currently I need features present in vtk (but not in vtk.js) such as VolumePicker and Multi-volume rendering. And I may have to load images (.vti files) of up to 100MB which can be rather laggy at the moment.
So I was trying to research on how I can maybe get a web assembly compiled version of vtk via emscripten with all the rendering done in vtk and not vtkjs. On the web itself, there should be some communication/event channel for the users to interact with the wasm components (like slicing through volumes, uploading image files, changing color map, etc.).
I managed to find the examples within the vtk repository but realised that the examples are not using Volumes. The examples also didn’t include how I could interact with the wasm components via javascript (from user input).
I also saw somewhere that volume rendering is not fully supported due to some opengl/shader issues (not sure if it has been resolved since as the threads were about 2 years ago).
However, due to my inexperience with dealing with wasm compilation and C++ in general, I’m not so sure if what I’m doing is possible + how to set and build properly.
Has anyone who done something similar could point me in the correct direction on how I could get started and whether the features I want are feasible? Thanks a lot!
I’m glad you are enjoying VTK.js.
Bringing VTK C++ features to VTK.js is indeed possible. You may want to look at how itk.wasm works and extend it to bring VTK into ITK.wasm.
However, please note that it won’t help you with “VolumePicker” and “Multi-volume” rendering.
Those features rely on the specific WebGL rendering in VTK.js and VTK C++ can’t be used for rendering (except if you totally get rid of VTK.js).
The only solution I can see is to properly add those features in VTK.js. (if you have funding we can help).
Hi, thanks for the response! Does this mean that it could be done if I rewrote everything in VTK C++? Would those specific VTK C++ features work if everything was compiled to .wasm? Let’s say VolumePicker for instance.
Would those specific VTK C++ features work if everything was compiled to .wasm? Let’s say VolumePicker for instance.
I’m not sure, it depends on the backend you end up using. If you choose SDL, it may have some limitations. I’m not very familiar with this approach.
I believe it should be much easier to add VolumePicking in VTK.js by adding WebGL code (and maybe get some inspiration by looking at how VTK C++/OpenGL has done so)…
You’re right. Thanks. I have looked into how I would do the features in C++, and I think the endeavour is probably not worth it. Will see how I can add VolumePicking in VTK.js. If it works well, I might just extend and open a PR.
I have written an application that is exclusively using VTK wasm, without vtk.js. The user interaction for moving, panning, zooming is through the VTK wasm, but user requests such as changing color maps, etc…, I pass from javascript to wasm through the use of integers and strings. It was too difficult to pass objects back and forth. I was able to easily compile VTK and my VTK app to wasm by following the guide.
I am currently just displaying textured polydata, but when I get time I was going to test the VTK wasm volume rendering capabilities.