vtkRemoteView get pixel intensity, how far the clident side interactorstyle can go

Web
1

Hi,

I am working on vtk.js client code for vtk remote rendering to get pixel density of image slice at mouse pressed location

  • For client side rendering, I can do everything on client side,

     interactor.onLeftButtonPress((ev) => {
    const position = ev.position;

    picker.pick([position.x, position.y, 0.0], renderer);
    const pickedPoint = picker.getPickPosition();


    
  

    // const value = dataArray.getScalarValueFromWorld([pickedPoint[0], pickedPoint[1], pickedPoint[2]])
    const value = this.imageData.getScalarValueFromWorld([pickedPoint[0], pickedPoint[1], 0])}

  • For remote rendering, I want to ask whether I can get pixel intensity only using client side programming. (Which shouldn’t be possible since client doesn’t have volume data, curious how far client code can go)Now, I can get event position, but seems picker got error:

    view.getInteractor().onLeftButtonPress((event) => {
    // Get the mouse position
    const x = event.position.x;
    const y = event.position.y;

    
    // 
    const interactor = this.view.getInteractor();
    const renderer = interactor.getCurrentRenderer()
    
    picker.pick([x, y, 140], renderer)
    const pickedPoint = picker.getPickPosition();

}

  • Since I already got the event position, I know if I pass this event to server side using rpc call, I certainly can got the pixel intensity, just curious whether how far I can go from client side
2

Like you said, the client does not have the data. Therefore, the client has to send the display (x,y) so the server can perform that lookup.

3

Hi, @Sebastien_Jourdain :grinning:

  • on server side, if I get event location using server sider interactor, I got the correct pixel index:

     event_pos = renderWindowInteractor.GetEventPosition()
 vc = vtk.vtkCoordinate()
 vc.SetCoordinateSystemToViewport()
 vc.SetValue((event_pos[0], event_pos[1], 0))
 pickPosition = list(vc.GetComputedWorldValue(renderer))

  • But if I use the x,y postion I got from vtk.js client, use this x,y to vc.SetValue(x, y, 0), the pixel index I got is wrong. How do set vtkCoornate to use the position obtained from client.
    vtkRemoteView.getInteractor().onLeftButtonPress((event) => {
    // Get the mouse position
    const x = event.position.x;
    const y = event.position.y;
    }

4

Also, I noticed, using
vc = vtk.vtkCoordinate()
vc.SetCoordinateSystemToViewport()
vc.SetValue((event_pos[0], event_pos[1], 0))
pickPosition = list(vc.GetComputedWorldValue(renderer))

some times the intensity value I retrieved is not correct( I have no way to verify the coordinate I obtained is correct or not), can only verify by checking the intensity value.

If I use vtkPointPicker, then the intensity I obtained seems all correct.

Any hints?

5

The origin is swap along Y between VTK/C++ and JavaScript events.
So you need to do (height - event.y)

6

Another factor could be the pixelRatio.

To double check you should place a 3D object in the scene that follow the display coord you get. That way you can be sure things align.

7

thank you for the hint. Indeed, I found out in my case, there are two different window size, one is client size html elment width and heigh. Another is server side, renderwindow.GetSize(). If x, y postion on client’s side was adjusted for this factor, then it woks as same as GetEventPostion on server side:
@exportRpc(“image.slice.pixel.intensity”)
def image_slice_pixel_intensity(self, x, y, element_w, element_h):
global renderer, renderWindow, renderWindowInteractor, cone, mapper, actor, textActor,
image_data
window_size = renderWindow.GetSize()
self.x = x/element_wwindow_size[0]
self.y = y/element_hwindow_size[1]

           event_pos = renderWindowInteractor.GetEventPosition()
           event_pos = (self.x, self.y)