TRAME dynamic thumbnail

Support
1

Hi, I’d like to embed a dynamic thumbnail of an iframe (used to display different Trame apps) within my larger application frontend. The idea is that whenever my iFrame rerenders, or whenever callbacks are triggered within the trame widgets (could be trame-vtk or other), I’d like to update an image that serves as a thumbnail of the actual viewer elsewhere in the interface.

What would be the most efficient and general way to achieve this? I can think of few possible directions:

One option would be to take screenshots on the backend and send them to the frontend via a WebSocket connection. This approach seems reasonable for VTK, but less so for JS-native widgets like Plotly.

Another option would be to capture the screenshot directly in the frontend using html2canvas, but this appears to conflict with WebGL widgets such as vtk.js.

Finally, I’ve considered using another Trame iframe to expose the image from the backend, but this feels a bit excessive in terms of resource usage.

I’d greatly appreciate any suggestions or guidance in the right direction. Thanks!

2

Hello,

What have you tried so far in terms of code?

best,

PC

3

It is hard to summarize the actual implementations here, as there were few and they were a bit lengthy, but in essence:

  • Subscribe to websocket for thumbnails in frontend. Take screenshot with VTK in backend and send it via websocket subscription. It somewhat works, but as I have multiple viewers this can become a bit messy. Also, when I want to use TRAME with other widgets, they all need their own screenshot implementation. For some this needs to happen through python within the frontend (kaleido for plotly), so I make a full circle. I therefore assumed it would be best to directly take the screenshot in the frontend from the iFrame, as this should also in principle work the same, no matter which TRAME widget is used.

  • I did that, enforcing a rerender of the img thumbnail on each click/drag mouse interaction with the iFrame. It seems to work fine for some widgets (plotly for instance), but for VTK, which is essential, I cannot get a screenshot (it is white) and the attempt of taking the screenshot seems to break my VTK implementation (the main actor is not rendered). I believe that this has to do with interference of the screenshot library (html2canvas) with WebGL in vtk.js.

  • What I have not tried yet, is using TRAME for this feature as well. In principal I could use the image-tools widget to show a thumbnail that is taken in the backend, but I think this could be quite a large overhead to run several servers, or several viewers on one server, just to display thumbnails, or is this may be negligible?

In the end of the day I don’t ask for a concrete instruction, but more a direction in case you clearly know which architecture would be best. Thanks!