I’m currently developing a web platform utilizing a simple variation of “remote” VTK rendering for volume visualiziation/interaction (C#) with rather complex VTK algorithms (C++) to achieve the goals of my project. I incorporated a large number of different VTk concepts, filters, pipelines within C++ and execute those function through PInvoke from my C# webserver.
The current remote rendering prinicple relies on rendering offscreen images on the server (into JPEG format) and sending those base64 encoded images over websocket to the client eventually displaying them within a HTML element.
I want to port this principle to a more stable solution with a VTK web server and VTK js on the client for VTK interaction.
I’m kinda struggling to fetch all the information needed to deploy such a solution. i’m still wondering if it’s somehow possible to setup a VTK server with my current C++ implementation as the core module (slight changes in implementation will have to be made, but i dont want to rewrite all my C++ code).
I’ve read something about VTK as a server within python. But that means that I cannot use my C++ VTK code and that I would have to rewrite it with the VTK python bindings?
Can you give me any kind of information which would help me figuring out which kind of infrastructure/concept/solution i should focus on to achieve a remote VTK renderer with client side interaction, ideally keeping most of my C++ code as it is?
Not sure how intensive your rendering is, but I am using VTK C++ compiled to WASM which is running the rendering and interacting on the client side browser, which is working very well. I had a Windows executable version of my application utilizing the VTK C++ libraries and I was able to reuse 95% of that code to compile to WASM and run on the clients web browser. Check out this link:
Thanks @Donny_Zimmerman for your answer.
So if I understand that correctly, your code is running solely on client side, with no remote rendering server?
My application is rendering rather big datasets (ranging from high-resolution cranial scans to full lower body CT scans). As I want to interact with the rendering from everyday devices (low-end desktops, Tablets, Smartphones) I’m not sure if I can run and execute all my rendering code from the client.
The way to go (I think) is to use some kind of VTK server (remote rendering) with VTK.js at client side visualizing and interacting with the render window. But I only found vtkWeb which uses python as a server. Is there a way to combine vtkPython with my C++ VTK implementation? I’ve looked through the python bindings and not all of the capabilities of VTK (that I am using right now) are wrapped with python so far - I kinda don’t want to miss the complexity/flexibility of C++ VTK
Yes, my application runs fully on the client browser, including rendering and interaction. Unfortunately I won’t be able to help you with server side rendering questions. Your dataset may in fact be too intensive to render on the client side, but WASM is a somewhat compiled language so it runs quite a bit faster than javascript in a browser. My application renders and interacts just fine on a smartphone. It was fairly easy for me to set up the WASM compilation environment and get my code rendering by following the instructions in the previous link I provided. You may find that it renders your dataset at an acceptable rate. The examples in the following link are rendering on the client side using vtk.js, which is a javascript implementation of the VTK library. I chose to use the C++ implementation compiled to WASM instead of vtk.js because not all VTK functions are implemented in vtk.js yet and also the speed and code reuse.
The Python server should work well because all VTK classes are Python-wrapped and you can wrap your C++ classes as well. If your classes are VTK-based then you can use VTK’s Python wrapper.
If you are not comfortable using Python then you can try Paraview render server. Or put together a simple rendering server and communicate with it using json-rpc or zmq (or if frame rate is not critical, then even web requests).
Sorry for the late reply but like @lassoan is saying your VTK/C++ should be able to be reused as-is. It is true that you will have to create a Python front-end to your code but with the wrapping available within VTK you should be able to drive your C++ code fairly easily and only focus on what the client (web/html) needs to trigger on the server side. You can even encapsulate all your C++ calls inside a single C++ class that will only expose in Python (via vtk/python wrapping) what you need from the web side.
A project that you may want to look at is PyWebVue (next gen VTK/ParaView-Web) and that VTK example doing remote rendering
The example readme suggest to use the code from the repo but it might be easier to use the published version.
Hi,
I am new to this world. I just go to the github and try to run the example : ./examples/01_cone/cone-pv-remote-rendering. I read the README.md and did what it says:
Yes I am running on Linux host.
If I do export DISPLAY=:1, then the app crashed
if I do not export DISPLAY, then it shows up cone
But i can not access the link on browser: localhost:1234, it shows forbidden
403: Forbidden
This is the debug mode
(py-lib) #( 08/10/21@ 5:09PM )( phongtd@nguyenth ):/tmp/py-lib
python …/py-web-vue/examples/01_cone/cone-vtk-remote-rendering/app.py --port 1234 --debug
DEBUG:asyncio:Using selector: EpollSelector
INFO:root:awaiting runner setup
CRITICAL:root:wslink: Starting factory
INFO:root:awaiting site startup
INFO:root:awaiting running future
INFO:aiohttp.access:127.0.0.1 [10/Aug/2021:10:09:27 +0000] "GET / HTTP/1.1" 403 178 “-” “Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36”
I’m not sure how you are getting asyncio as backend. That is probably because I did not fixed wslink dependency version in the requirement of pywebvue.
Can you try to update your virtual environment to use wslink==0.1.14. Soon enough pywebvue will use the latest version of wslink, but right now it is not compatible.
Thanks Sebastien, it works with wslink==0.1.14. I have a question, I open 2 separate browsers (from 2 different clients), and access the same url: :1234. Then they show up the same content. It does mean that anything changing from one client will reflect on the other. So does the library “py-web-vue” supports each client has it own session ? So each time one client connects to the server, server will only process separately for that client ?
So what I understand that each time client connects to the server via wslink, server should spawn a new process to serve that client on a separate wslink. And each process should run on a different port. Does this py-web-view support a pool of port so that each time client connects, a port will be allocated ?
Yes just add --launcher and your app will support multi-clients. But for a real deployment, it would be best to use apache to expose your app behind a single port and do the ws forwarding to the proper process behind the scene.
Thanks a lots Sebastien,
May I ask another question. I want to implement a 3D remote rendering system. Server will render 3D images and only returns 2D images to the client. So client only gets 2D images instead of whole 3D images. In the example : examples/03_applications/vtk-f1-picking/app.py), I see that client still consumes the while 3D images (f1.vti), any only communicate with server with points. Do you have an example of illustrating server returning 2D images each time client calls (for example clients calls rotate object, cut objects, …) ?
The only tricky part right now is that we are transitioning network library inside PyWebVue and VTK/PV are lagging behind. Down the road everything will be integrated and easy but right now, you need to hand pick your various versions. See that section for more details
Thanks Sebastien,
I managed to run the example with Para and VTK backend. In the example: 02_contour/pv-toggle-rendering, I saw that server returns the 2D images to client for displaying. I tried to replace the file head.vti by another file which is ~ 200 MB (which I converted from dicom files with tool gdcm2vtk: gdcm2vtk 3_Thorax output.vti). The performance of server is quite terrible. Server acts quite slowly and client had to wait to get 2D images quite slow. I noticed there are some errors from the servers. Do you have any idea on how to boost the performance (my desktop is Intel(R) Core™ i5-8400 CPU @ 2.80GHz, RAM 16 Gb).
DEBUG:root:wslink incoming msg {“wslink”:“1.0”,“id”:“rpc:cfb5ae80dad2d4242a901216396feb92f:56”,“method”:“viewport.image.push”,“args”:[{“view”:“5797”,“size”:[912,613]}],“kwargs”:{}}
DEBUG:root:wslink incoming msg {“wslink”:“1.0”,“id”:“rpc:cfb5ae80dad2d4242a901216396feb92f:57”,“method”:“ws.vue.trigger”,“args”:[“view.camera”,,{}],“kwargs”:{}}
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name=‘Task-74’ coro=<WslinkHandler.sendWrappedMessage() done, defined at /tmp/py-env/lib/python3.8/site-packages/wslink/backends/aiohttp/init.py:392> exception=AssertionError()>
Traceback (most recent call last):
File “/tmp/py-env/lib/python3.8/site-packages/wslink/backends/aiohttp/init.py”, line 439, in sendWrappedMessage
await ws.send_bytes(attachments[key])
File “/tmp/py-env/lib/python3.8/site-packages/aiohttp/web_ws.py”, line 307, in send_bytes
await self._writer.send(data, binary=True, compress=compress)
File “/tmp/py-env/lib/python3.8/site-packages/aiohttp/http_websocket.py”, line 685, in send
await self._send_frame(message, WSMsgType.BINARY, compress)
File “/tmp/py-env/lib/python3.8/site-packages/aiohttp/http_websocket.py”, line 656, in _send_frame
await self.protocol._drain_helper()
File “/tmp/py-env/lib/python3.8/site-packages/aiohttp/base_protocol.py”, line 84, in _drain_helper
assert waiter is None or waiter.cancelled()
AssertionError
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name=‘Task-77’ coro=<WslinkHandler.sendWrappedMessage() done, defined at /tmp/py-env/lib/python3.8/site-packages/wslink/backends/aiohttp/init.py:392> exception=AssertionError()>
Traceback (most recent call last):
File “/tmp/py-env/lib/python3.8/site-packages/wslink/backends/aiohttp/init.py”, line 439, in sendWrappedMessage
await ws.send_bytes(attachments[key])
File “/tmp/py-env/lib/python3.8/site-packages/aiohttp/web_ws.py”, line 307, in send_bytes
await self._writer.send(data, binary=True, compress=compress)
File “/tmp/py-env/lib/python3.8/site-packages/aiohttp/http_websocket.py”, line 685, in send
await self._send_frame(message, WSMsgType.BINARY, compress)
File “/tmp/py-env/lib/python3.8/site-packages/aiohttp/http_websocket.py”, line 656, in _send_frame
await self.protocol._drain_helper()
File “/tmp/py-env/lib/python3.8/site-packages/aiohttp/base_protocol.py”, line 84, in _drain_helper
assert waiter is None or waiter.cancelled()
AssertionError
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name=‘Task-73’ coro=<WslinkHandler.sendWrappedMessage() done, defined at /tmp/py-env/lib/python3.8/site-packages/wslink/backends/aiohttp/init.py:392> exception=RuntimeError(‘dictionary changed size during iteration’)>
Traceback (most recent call last):
File “/tmp/py-env/lib/python3.8/site-packages/wslink/backends/aiohttp/init.py”, line 422, in sendWrappedMessage
for key in attachments:
RuntimeError: dictionary changed size during iteration
