Performance drop after usage (wslink + vtk)

Web
1

Hi again :slight_smile:

I am building a web application with wslink + vtk (python wrapper).

I am rendering a fairly big model on the server, everything starts smooth and the model renders correctly with an acceptable number of FPS.

After just utilising rotating it around the performance starts to drop and everything on the server gets very slow. Every message any new request is now much slower and it never recovers.

Is there a cache being created that could be cleaned? I am not sure what I can pursue here.

I already am setting interactive ratios and interactive quality but the key thing is the more events I send (using mouse interactions) over time the slower it gets, and it never recovers.

2

Wanted to add that this occurs (either only or at least much faster) when more than one view (render_window) is created in the server.

I am actively unregistering views not on display.

The images from other views are not being sent but they are affecting.

Here’s a minimalist server code:

class MyProtocols(vtkWebProtocol):
    started = False

    def render(self, data: vtk.vtkMultiBlockDataSet):
        """
        Render the data.
        """
        self.render_window = vtk.vtkRenderWindow()
        self.renderer = vtk.vtkRenderer()
        self.render_window.AddRenderer(self.renderer)
        self.interactor = vtk.vtkRenderWindowInteractor()
        self.interactor.SetInteractorStyle(vtk.vtkInteractorStyleTrackballCamera())
        self.interactor.SetRenderWindow(self.render_window)
        self.render_window.SetOffScreenRendering(1)

        self.actor = vtk.vtkActor()
        self.mapper = vtk.vtkCompositePolyDataMapper()
        self.actor.SetMapper(self.mapper)
        self.mapper.SetInputDataObject(data)
        self.renderer.AddActor(self.actor)

        self.render_window.Render()

        return self.render_window

    @register("start")
    def start(self):
        if self.started:
            return
        self.started = True

        data = load_data()
        view_ids = []

        for _ in range(0, 4):
            rw = self.render(data)
            view_id = self.getGlobalId(rw)
            view_ids.append(view_id)

        return view_ids

React (remote render code):

import React, { useEffect, useLayoutEffect, useRef, useState } from "react";
import vtkRemoteView from "@kitware/vtk.js/Rendering/Misc/RemoteView";
import vtkViewStream from "@kitware/vtk.js/IO/Core/ImageStream/ViewStream";

interface RemoteViewerProps {
  client: any;
  viewStream: vtkViewStream;
  registerViewStream: (viewStream: vtkViewStream) => void;
  unregisterViewStream: (viewStream: vtkViewStream) => void;
}

export const RemoteViewer = React.memo(
  ({
    viewStream,
    client,
    registerViewStream,
    unregisterViewStream,
  }: RemoteViewerProps) => {
    const ref = useRef<HTMLDivElement>(null);
    const [remoteView, setRemoteView] = useState<vtkRemoteView | null>(null);
    let result: React.ReactNode;

    useEffect(() => {
      if (viewStream) {
        // Unregister first to prevent duplicates
        unregisterViewStream(viewStream);
        // The register method does not check for existing viewStreams
        registerViewStream(viewStream);
      }

      const setup = async () => {
        if (
          typeof window !== undefined &&
          client &&
          ref.current &&
          !remoteView
        ) {
          const session = client.getConnection().getSession();

          const vtkRemoteView = await import(
            "@kitware/vtk.js/Rendering/Misc/RemoteView"
          );
          const newView = vtkRemoteView.newInstance({
            rpcWheelEvent: "viewport.mouse.zoom.wheel",
            viewStream,
          });

          newView.setSession(session);
          newView.setInteractiveRatio(0.9); // the scaled image compared to the clients view resolution
          newView.setInteractiveQuality(70); // jpeg quality

          newView.setContainer(ref.current);
          newView.resize();
          window.addEventListener("resize", newView.resize);
          ref.current.addEventListener("resize", newView.resize);

          setRemoteView(newView);
        } else if (!client && remoteView) {
          setRemoteView(null);
        }
      };

      setup();

      return () => {
        if (viewStream) {
          unregisterViewStream(viewStream);
        }
      };
    }, [ref, client, remoteView, viewStream]);


    if (client) {
      result = (
        <div
          style={{
            width: "100%",
            height: "100%",
            overflow: "hidden",
            position: "relative",
            background: "none",
          }}
          ref={ref}
        />
      );
    }

    return result;
  }
);

RemoteViewer.displayName = "RemoteViewer";
3

I think it might be related to vtkWebMouseHandler and vtkWebPublishImageDelivery in VTK protocols.

I am not sure if this is the only problem or the one causing, but it seems that the action property on the event is always down which in turn creates a StartInteractionEvent multiple times before calling finally EndInteractionEvent.

I’ve noticed that viewsInAnimation does not get cleared as intended, or at least to my level of understanding. It seems that we push the same viewId over and over to the viewsInAnimation array until it gets cleared only 1 times on the end event.

This after some time from my understanding is causing degradation in the streaming protocol.

I am not 100% sure on how the code works so it could be on purpose but it seems like this is causing a leak which in turn forces more renderings than necessary.

4

Again without knowing 100% of the code I think the vtkRemoteView with its interactorStyle vtkInteractorStyleRemoteMouse sends Start and End events correctly on it’s own, i.e. StartInteractionEvent and EndInteractionEvent. So it could be that without this bit:

if event["action"] == "down":
    self.getApplication().InvokeEvent("StartInteractionEvent")

if event["action"] == "up":
    self.getApplication().InvokeEvent("EndInteractionEvent")

in

class vtkWebMouseHandler(vtkWebProtocol):
    @exportRpc("viewport.mouse.interaction")
    def mouseInteraction(self, event):

is actually irrelevant and in this case causing the leak, but I’m not sure if it’s being used elsewhere, what I know here is that action is always down until it is up thanks to the way the interactor is created:

function createRemoteEvent(callData) {
    // Fields:
    //  - action: [down, up]
    //  - x, y (Normalized)
    // Flags:
    //  - buttonLeft, buttonMiddle, buttonRight
    //  - shiftKey, ctrlKey, altKey, metaKey
    const {
      buttonLeft,
      buttonMiddle,
      buttonRight
    } = model;
    const shiftKey = callData.shiftKey ? 1 : 0;
    const ctrlKey = callData.controlKey ? 1 : 0;
    const altKey = callData.altKey ? 1 : 0;
    const metaKey = callData.metaKey ? 1 : 0; // Might be platform specific
    const action = buttonLeft || buttonMiddle || buttonRight ? 'down' : 'up';

    // Fixme x / y
    const [width, height] = model._interactor.getView().getSizeByReference();
    let {
      x,
      y
    } = callData.position;
    x /= width;
    y /= height;
    return {
      action,
      x,
      y,
      buttonLeft,
      buttonMiddle,
      buttonRight,
      shiftKey,
      altKey,
      ctrlKey,
      metaKey,
      ...model.remoteEventAddOn
    };
  }

As we can see there action is always down or up causing the beforementioned problem.

5

Can you reproduce that issue in a small trame example? That might be easier to track down and fix.