I am struggling a bit with multiple render windows for a WASM application. It appears to me that the CanvasId is hard-coded to canvas and VTK only supports a single render window for WASM?
Is this the case? We are considering reaching out for professional help. Currently, I am in a situation where my debugging facilities are much better at home, so writing here using my private account.
Thanks in advance
Jens Munk
Hello @Jens_Munk_Hansen
It appears to me that the
CanvasIdis hard-coded tocanvasand VTK only supports a single render window for WASM?
That is correct.
The limitation really came from the lack of support in SDL2’s emscripten code. I’ve lifted the limitation by updating VTK to use the low level emscripten’s HTML5 API in https://gitlab.kitware.com/vtk/vtk/-/commit/540986cdb3be2e5e35edc6717d431d9acb2e8abf and https://gitlab.kitware.com/vtk/vtk/-/commit/4b0b3b317d70c605d8b0b2dd2b88f69f64e788d9. That should technically allow VTK to draw inside canvas element in the DOM whose ID is not “canvas”
Although, support for multiple render windows in VTK.wasm is untested territory. Perhaps your application could be a good testing candidate for us! Please do not hesitate to contact us for further assistance regarding this.
I will talk to my manager tomorrow. We only have 10 support hours, but we could buy more, I really hope so.
Now, I figured out that in our case, the name #canvas in vtkWebAssemblyOpenGLRenderWindow is unrelated to the fact that I get a null, when querying using :
<HTMLCanvasElement>document.getElementById('anotherName')
The name canvas is from some auto-generated glue code from Emscripten. Emscripten is open-source, so this I should figure out. Once, we get a proof-of-concept, I really hope that we will reach out for some ongoing support with wrapping etc.
Thanks again and keep up the good work
Jens
I managed to get things working with multiple windows sharing an interactor, but I really don’t like this approach. I have also noticed some weird VTK code that gets injected into the WASM. Manager currently on holidays, so we will get things up and running. For sure, I would recommend that for our final product we get support from you. It is not so nice maintaining an in-house integration of VTK for a window manager. I lived with my own Qt/QML integration two year before it was included in VTK.
I have also noticed some weird VTK code that gets injected into the WASM.
Can you share what that looks like? It may be the new helper code from vtkWebAssemblyRenderWindowInteractor.js
Sure it is the stuff for the VTKCanvas.
var VTKCanvas = {
initializeCanvasElement: (elementId, applyStyle) => {
const canvasElem = findCanvasEventTarget(elementId, applyStyle);
if (!canvasElem) {
return;
}
const containerElem = canvasElem.parentElement;
const body = document.querySelector("body");
if (applyStyle) {
if (body === containerElem) {
body.style.margin = 0;
body.style.width = "100vw";
body.style.height = "100vh";
} else {
containerElem.style.position = "relative";
containerElem.style.width = "100%";
containerElem.style.height = "100%";
}
canvasElem.style.position = "absolute";
canvasElem.style.top = 0;
canvasElem.style.left = 0;
canvasElem.style.width = "100%";
canvasElem.style.height = "100%";
}
},
getParentElementBoundingRectSize: elementId => {
const canvasElem = findCanvasEventTarget(elementId);
if (!canvasElem) {
return 0;
}
const containerElem = canvasElem.parentElement;
const dpr = window.devicePixelRatio;
const width = containerElem.getBoundingClientRect().width;
const height = containerElem.getBoundingClientRect().height;
const w = Math.floor(width * dpr + .5);
const h = Math.floor(height * dpr + .5);
const sizePtr = _malloc(8);
const idx = ((sizePtr) >>> 2);
HEAP32.set([ w, h ], idx >>> 0);
return sizePtr;
}
};
It look super arbitrary that we write the height and width to the heap and shift is 0 bits. You are right, it is the new helper code from the vtkWebAssemblyRenderWindowInteractor.js.
Also, would it be a good idea to expose to JavaScript, vtkInitializeCanvasElement and use this for creating additional canvases and perhaps vtkGetParentElementBoundingRectSize for resizing canvases when resizing a canvas.
At the moment, we are starting and stopping the event loop in JavaScript and reusing a single interactor for two windows, changing the canvasId of course.
I borrowed some of your JavaScript wrapping code to make it easier for experimenting.
It look super arbitrary that we write the height and width to the heap and shift is 0 bits. You are right, it is the new helper code from the
vtkWebAssemblyRenderWindowInteractor.js.
That library code is an implementation detail in VTK. This is how it communicates the size available for rendering to VTK. In the event handler in C++, we decode the contents of the returned pointer into width and height.
Also, would it be a good idea to expose to JavaScript,
vtkInitializeCanvasElementand use this for creating additional canvases and perhapsvtkGetParentElementBoundingRectSizefor resizing canvases when resizing a canvas.
Hmm, those two are already javascript functions and could be accessible on the wasm instance. Can you try searching the generated JS for those?
You are right, the are available on the WASM interface. Thank you for reaching out.
I will push for that you get involved. The wrapping and selecting of what to include in the different .wasm files is a little invasive on our VTK branch. Trying to establish a runtime containing VTK/Common, but without windows. I imagine having some .wasm modules compatible with your data model without knowing anything about rendering.
My current solution for an interactor that can be used for multiple windows, I have made using vtkGenericRenderWindowInteractor and is shown below.
export class JSVTKCanvasHandler {
// Timer to control responses
Timer = class {
constructor(callback, interval) {
this.callback = callback;
this.interval = interval;
this.timerId = null;
}
start() {
if (this.timerId === null) {
this.timerId = setTimeout(() => {
this.callback();
this.timerId = null; // Reset the timerId after the callback is executed
}, this.interval);
}
}
stop() {
if (this.timerId !== null) {
clearTimeout(this.timerId);
this.timerId = null;
}
}
restart() {
this.stop();
this.start();
}
}
static MouseButton = {
LEFT: 0,
MIDDLE: 1,
RIGHT: 2
};
static mouseButtonEvents = {
LEFT: 'LeftButtonReleaseEvent',
RIGHT: 'RightButtonReleaseEvent',
MIDDLE: 'MiddleButtonReleaseEvent'
};
static vtkToJsCursorMap = {
0: 'default', // VTK_CURSOR_DEFAULT
1: 'default', // VTK_CURSOR_ARROW
2: 'default', // VTK_CURSOR_SIZENE
3: 'default', // VTK_CURSOR_SIZENWSE
4: 'default', // VTK_CURSOR_SIZESW
5: 'default', // VTK_CURSOR_SIZESE
6: 'ns-resize', // VTK_CURSOR_SIZENS
7: 'ew-resize', // VTK_CURSOR_SIZEWE
8: 'pointer', // VTK_CURSOR_SIZEALL
9: 'pointer', // VTK_CURSOR_HAND
10: 'crosshair' // VTK_CURSOR_CROSSHAIR
}
constructor(canvas, iren, wasmExports) {
this.canvas = canvas;
// The VTK interactor. TODO: Figure out which interactors that require a timer
this.iren = iren;
this.handleMouseDown = this.handleMouseDown.bind(this);
this.handleMouseMove = this.handleMouseMove.bind(this);
this.handleMouseUp = this.handleMouseUp.bind(this);
this.handleDoubleClick = this.handleDoubleClick.bind(this);
this.handleKeyDown = this.handleKeyDown.bind(this);
this.handleEnter = this.handleEnter.bind(this);
this.handleLeave = this.handleLeave.bind(this);
this.handleTimer = this.handleTimer.bind(this);
this.handleWheel = this.handleWheel.bind(this);
this.handleResize = this.handleResize.bind(this);
this.changedCursor = this.changedCursor.bind(this);
this.showCursor = this.showCursor.bind(this);
this.addEventListeners();
this.vtk = wasmExports;
// TODO: Verify if lastPosition = (0,0) is an issue
this.lastPosition = {x: 0, y: 0};
this.lastModifiers = {ctrlPressed: false, shiftPressed: false};
this.wheelData = 0;
this.mouseState = {LEFT: false,
RIGHT: false,
MIDDLE: false};
this.mouseEnabled = true;
// Callback for creating timer
this.createTimerCommand = new this.vtk.vtkCallbackCommand();
this.createTimerCommand.SetCallback((caller, evId, clientData, callData) => {
this.createTimer();
});
// Callback for destroying timer
this.destroyTimerCommand = new this.vtk.vtkCallbackCommand();
this.destroyTimerCommand.SetCallback((caller, evId, clientData, callData) => {
this.destroyTimer();
});
this.cursorChangedCommand = new this.vtk.vtkCallbackCommand();
this.cursorChangedCommand.SetCallback((caller, evId, clientData, callData) => {
this.changedCursor();
});
this.iren.AddObserver('CreateTimerEvent', this.createTimerCommand, 0.0)
this.iren.AddObserver('DestroyTimerEvent', this.destroyTimerCommand, 0.0)
this.iren.GetRenderWindow().AddObserver('CursorChangedEvent',
this.cursorChangedCommand, 0.0)
// Timer, we allow mouse events every 10 ms
this.timer = new this.Timer(this.handleTimer, 10);
// For debugging purposes. To see how many mouse event we have
this.counter = 0;
}
addEventListeners() {
this.canvas.addEventListener('mousedown', this.handleMouseDown);
this.canvas.addEventListener('mousemove', this.handleMouseMove);
this.canvas.addEventListener('mouseup', this.handleMouseUp);
this.canvas.addEventListener("mouseenter", this.handleEnter, false);
this.canvas.addEventListener("mouseleave", this.handleLeave, false);
this.canvas.addEventListener('dblclick', this.handleDoubleClick);
this.canvas.addEventListener('wheel', this.handleWheel);
window.addEventListener('resize', this.handleResize);
document.addEventListener('keydown', this.handleKeyDown);
// Hide the pop-up menu
document.addEventListener('contextmenu', function(event) {
event.preventDefault();
});
}
createTimer() {
this.mouseEnabled = false;
this.timer.start();
}
destroyTimer() {
this.mouseEnabled = true;
this.timer.stop();
}
changedCursor() {
// Called when the CursorChangedEvent fires on the render window.
// This indirection is needed since when the event fires, the current
// cursor is not yet set so we defer this by which time the current
// cursor should have been set.
//
// ISSUE: Needs to be in-sync with the other timer, I guess.
setTimeout(this.showCursor, 10);
}
showCursor() {
const vtk_cursor = this.iren.GetRenderWindow().GetCurrentCursor()
const js_cursor = JSVTKCanvasHandler.vtkToJsCursorMap[vtk_cursor];
console.log('show cursor');
this.setCursor(js_cursor)
}
setCursor(cursorStyle) {
document.body.style.cursor = cursorStyle;
}
handleTimer() {
this.iren.TimerEvent();
}
getPixelRatio() {
return window.devicePixelRatio || 1;
}
getMouseButton(event) {
return event.button;
}
getMousePosition(event) {
// TODO: Do we need to flip up/down
const rect = this.canvas.getBoundingClientRect();
return {
x: (event.clientX - rect.left) * (this.canvas.width / rect.width),
y: (event.clientY - rect.top) * (this.canvas.height / rect.height)
};
}
areModifiersPressed(event) {
return {
ctrlPressed: event.ctrlKey | this.lastModifiers.ctrlPressed,
shiftPressed: event.shiftKey | this.lastModifiers.shiftPressed
};
}
setEventInformation(x, y, ctrl=false, shift=false, key=0, repeat=0, keysym='') {
const devicePixelRatio = this.getPixelRatio();
const scale = devicePixelRatio;
const flipUpDown = false;
if (flipUpDown) {
this.iren.SetEventInformation(Math.round(x*scale),
Math.round((this.canvas.height - y - 1)*scale), ctrl, shift, key, repeat, keysym);
} else {
this.iren.SetEventInformation(Math.round(x*scale),
Math.round(y*scale), ctrl, shift, key, repeat, keysym);
}
}
handleEnter(event) {
const modifiers = this.areModifiersPressed(event);
this.setEventInformation(this.lastPosition.x,
this.lastPosition.y,
modifiers.ctrlPressed,
modifiers.shiftPressed);
this.iren.EnterEvent();
}
handleLeave(event) {
const modifiers = this.areModifiersPressed(event);
this.setEventInformation(this.lastPosition.x,
this.lastPosition.y,
modifiers.ctrlPressed,
modifiers.shiftPressed, 0, 0, '');
for (const button in JSVTKCanvasHandler.mouseButtonEvents) {
if (this.mouseState[button]) {
this.iren[JSVTKCanvasHandler.mouseButtonEvents[button]]();
this.mouseState[button] = false;
}
}
this.currentCanvas = null;
this.iren.LeaveEvent();
}
handleResize() {
const containerElem = this.canvas.parentElement;
const body = document.querySelector('body');
const applyStyle = true;
if (applyStyle) {
if (body === containerElem) {
body.style.margin = 0;
body.style.width = '100vw';
body.style.height = '100vh';
} else {
containerElem.style.position = 'relative';
containerElem.style.width = '100%';
containerElem.style.height = '100%';
}
this.canvas.style.position = 'absolute';
this.canvas.style.top = 0;
this.canvas.style.left = 0;
this.canvas.style.width = '100%';
this.canvas.style.height = '100%';
}
const scale = devicePixelRatio;
this.iren.GetRenderWindow().SetDPI(Math.round(72*scale));
this.canvas.width = containerElem.clientWidth;
this.canvas.height = containerElem.clientHeight;
this.iren.GetRenderWindow().SetSize(this.canvas.width, this.canvas.height);
this.iren.ConfigureEvent();
this.iren.GetRenderWindow().Render();
}
handleWheel(event) {
event.preventDefault(); // Prevent the default scrolling behavior
this.wheelData += event.deltaY;
if (this.wheelData >= 120) {
this.iren.MouseWheelForwardEvent();
this.wheelData = 0;
}
else if (this.wheelData <= -120) {
this.iren.MouseWheelBackwardEvent();
this.wheelData = 0;
}
}
handleMouseMove(event) {
const rect = this.canvas.getBoundingClientRect();
if (
event.clientX >= rect.left &&
event.clientX <= rect.right &&
event.clientY >= rect.top &&
event.clientY <= rect.bottom
) {
this.currentCanvas = this.canvas;
this.counter = this.counter + 1;
// One could update every counter % 200 == 0 for styles
// which do not use timers for smooth motion
if (this.iren.GetInteractorStyle().GetUseTimers() == 0 || this.mouseEnabled) {
// Handle movement
const position = this.getMousePosition(event);
const modifiers = this.areModifiersPressed(event);
this.lastModifiers = modifiers;
this.lastButtons = this.getMouseButton(event);
this.lastPosition = position;
this.setEventInformation(position.x, position.y,
modifiers.ctrlPressed,
modifiers.shiftPressed,
0, 0, '');
this.iren.MouseMoveEvent();
}
} else {
console.log("canvas is null");
this.currentCanvas = null;
}
}
handleDoubleClick(event) {
if (this.currentCanvas) {
const mouseButton = this.getMouseButton(event);
const modifiers = this.areModifiersPressed(event);
const position = this.getMousePosition(event);
const repeat = 1;
this.iren.SetEventInformation(position.x, position.y, modifiers.ctrlPressed, modifiers.shiftPressed, 0, repeat, '');
switch (mouseButton) {
case JSVTKCanvasHandler.MouseButton.LEFT:
this.iren.LeftButtonPressEvent();
// Otherwise, it spins like mad
this.mouseState.LEFT = true;
break;
case JSVTKCanvasHandler.MouseButton.MIDDLE:
this.iren.MiddleButtonPressEvent();
this.mouseState.MIDDLE = true;
break;
case JSVTKCanvasHandler.MouseButton.RIGHT:
this.iren.RightButtonPressEvent();
this.mouseState.RIGHT = true;
break;
default:
break;
}
}
}
handleKeyDown(event) {
if (this.currentCanvas) {
const key = event.key;
const asciiCode = key.length === 1 ? key.charCodeAt(0) : null;
const modifiers = this.areModifiersPressed(event);
if (asciiCode !== null) {
this.iren.SetEventInformation(this.lastPosition.x,
this.lastPosition.y,
modifiers.ctrlPressed, modifiers.shiftPressed, asciiCode, 0, key);
this.iren.KeyPressEvent();
this.iren.CharEvent();
} else {
console.log(`No ASCII code for this key`);
}
}
}
handleKeyUp(event) {
if (this.currentCanvas) {
// Create a getCharAndKeySym(event) function
const key = event.key;
const asciiCode = key.length === 1 ? key.charCodeAt(0) : null;
const modifiers = this.areModifiersPressed(event);
this.iren.SetEventInformation(this.lastPosition.x,
this.lastPosition.y,
modifiers.ctrlPressed, modifiers.shiftPressed, asciiCode, 0, key);
this.iren.KeyReleaseEvent();
}
}
handleMouseDown(event) {
if (this.currentCanvas) {
const mouseButton = this.getMouseButton(event);
const modifiers = this.areModifiersPressed(event);
const position = this.getMousePosition(event);
const repeat = 0; // Equal to 1 for double-click
this.iren.SetEventInformation(position.x, (this.canvas.height - 1 - position.y), modifiers.ctrlPressed, modifiers.shiftPressed, 0, repeat, '');
switch (mouseButton) {
case JSVTKCanvasHandler.MouseButton.LEFT:
this.iren.LeftButtonPressEvent();
this.mouseState.LEFT = true;
break;
case JSVTKCanvasHandler.MouseButton.MIDDLE:
this.iren.MiddleButtonPressEvent();
this.mouseState.MIDDLE = true;
break;
case JSVTKCanvasHandler.MouseButton.RIGHT:
this.iren.RightButtonPressEvent();
this.mouseState.RIGHT = true;
break;
default:
break;
}
}
}
handleMouseUp(event) {
if (this.currentCanvas) {
const mouseButton = this.getMouseButton(event);
const modifiers = this.areModifiersPressed(event);
const position = this.getMousePosition(event);
const repeat = 0; // Equal to 1 for double-click
this.iren.SetEventInformation(position.x, position.y, modifiers.ctrlPressed, modifiers.shiftPressed, 0, repeat, '');
switch (mouseButton) {
case JSVTKCanvasHandler.MouseButton.LEFT:
this.iren.LeftButtonReleaseEvent();
this.mouseState.LEFT = false;
break;
case JSVTKCanvasHandler.MouseButton.MIDDLE:
this.iren.MiddleButtonReleaseEvent();
this.mouseState.MIDDLE = false;
break;
case JSVTKCanvasHandler.MouseButton.RIGHT:
this.iren.RightButtonReleaseEvent();
this.mouseState.RIGHT = false;
break;
default:
break;
}
}
}
}
When it is a bit more tested, it may be a candidate for Wrapping/JavaScript/