coordinate transformation when screen scaling is not 1.0

Support
1

Hi there!

I am working on a custom ImageInteractorStyle derived from vtkInteractorStyleImage to move an image (or camera in this case) with the left mouse button pressed. I also noticed that the default implementation doesn’t seem to support screen scaling other than 1.0, since the image doesn’t move exactly with the mouse cursor. My override implementation of OnMouseMove looks like this:

void ImageInteractorStyle::OnMouseMove()
{
if (m_panning && this->GetCurrentRenderer()) {

    // Get the current mouse position
    int currPos[2];
    this->Interactor->GetEventPosition(currPos);

    // Convert the initial and current mouse positions to world coordinates
    vtkSmartPointer<vtkCoordinate> coordinate1 = vtkSmartPointer<vtkCoordinate>::New();
    vtkSmartPointer<vtkCoordinate> coordinate2 = vtkSmartPointer<vtkCoordinate>::New();
    
    coordinate1->SetCoordinateSystemToViewport();
    coordinate2->SetCoordinateSystemToViewport();
    
    coordinate1->SetViewport(this->GetCurrentRenderer());
    coordinate2->SetViewport(this->GetCurrentRenderer());
    
    coordinate1->SetValue(this->m_initialMousePos[0], this->m_initialMousePos[1], 0);
    double* initialWorldPos = coordinate1->GetComputedWorldValue(this->GetCurrentRenderer());
    
    coordinate2->SetValue(currPos[0], currPos[1], 0);
    double* currentWorldPos = coordinate2->GetComputedWorldValue(this->GetCurrentRenderer());
    
    // Calculate the offset in world coordinates
    double deltaX = currentWorldPos[0] - initialWorldPos[0];
    double deltaY = currentWorldPos[1] - initialWorldPos[1];

    // Move the camera to follow the panning
    vtkCamera* camera = this->GetCurrentRenderer()->GetActiveCamera();
    // Update the camera position and focal point
    double newCameraPosition[3] = { m_initialCameraPos[0] - deltaX,
        m_initialCameraPos[1] - deltaY, m_initialCameraPos[2] };
    double newCameraFocalPoint[3] = { m_initialFocalPos[0] - deltaX,
        m_initialFocalPos[1] - deltaY, m_initialFocalPos[2] };
    
    camera->SetPosition(newCameraPosition);
    camera->SetFocalPoint(newCameraFocalPoint);

    // Forward events
    vtkInteractorStyleImage::OnMouseMove();

}

This behaves like the standard image interactor style, except that it now moves with the left mouse button pressed.

To make the movement of the image or camera exactly match the movement of the mouse, I have to multiply the delta by the current screen scale.

deltaX *= 1.25;
deltaY *= 1.25;

I had a similar problem when working on a custom CubeInteractorStyle. There I got the scaling from a QQuickItem which I use to render the vtk content.

Is there any other way to account for different screen scaling within the vtk coordinate transformation system??

I have also tried using vtkRenderWindow::GetDeviceToWorldMatrixForDevice but it only gives me the identity matrix.

2

Apologies in advance for taking this off-track, but I just wanted to verify that the viewport coords were being handled correctly before getting into the screen scaling.

When you set viewport coordinates, you set the “depth” to zero:

coordinate2->SetValue(currPos[0], currPos[1], 0);

This means that vtkCoordinate will give you the equivalent world position at the near clipping plane, rather than at the focal plane or wherever it is that the image is located. If you’re using a parallel projection then this is fine, but if you’re using a perspective projection, then it’s necessary to use the correct depth coordinate or else you’ll end up with a scaling error.

coordinate1->SetValue(this->m_initialMousePos[0], this->m_initialMousePos[1], depth);
coordinate2->SetValue(currPos[0], currPos[1], depth);

Of course I might be way off in my assumptions here, e.g. if you’re using parallel projection and not perspective projection.

3

Please ignore the incorrect depth math from my last reply. A correct method for computing depth can be found in the vtkInteractorStyleTrackballCamera::Pan() method, which feeds the camera FocalPoint through a world-to-display or world-to-viewport coordinate transformation.

4

Thanks for pointing that out. I am actually using parallel projection.

Using the code from vtkInteractorStyleTrackballCamera::Pan() gives me the same offset motion. Multiplying the motionVector by the screen scale factor fixes this. But I still think there should be a vtk internal way to handle this.

5

The thing is, VTK does handle the scale factor, but at this point in the code (in the IteractorStyle classes) it should already have been applied.

So when you do this, it should already be giving you correctly scaled coordinates that you can use for interaction:

this->Interactor->GetEventPosition(currPos)

If that isn’t giving you correctly scaled coordinates, then some problem is happening in the way that mouse coordinates are being provided to the interactor. And the code that provides the coords to the interactor is all platform-specific.

For example, see the way scaling is applied in QVTKInteractorAdapter via DevicePixelRatio.

In your case, what is providing the pixel coords to the interactor? What platform, GUI toolkit, etc. are you using?

6

I am using QQuickVTKItem which manages VTK rendering in the QML scenegraph as I am working on a QtQuick application. So the GUI framework is Qt/QML. My development environment is Windows 11, but Linux/MacOs are also build targets.

Exactly, in the QVTKInteractorAdapter the DevicePixelRatio is set as a member and is used with every position related calculation.

7

I just found out that the bug was fixed 3 months ago QQuickVTKItem: Scale event position when screen is scaled. I will update vtk and the problem should be gone. Thanks for your help!