Bug in vtkCamera::setUserViewTransform with an inverse vtkTransform

Hi,
I think I found a bug somewhere between vtkCamera and vtkAbstractTransform/vtkTransform that leads to a deadlock due to a mutex being locked twice.
I made a minimal example: just use the vtkCamera example and add two lines just after creating the vtkCamera:

vtkNew<vtkTransform> myViewTransform; // Identity Transform
camera->SetUserViewTransform(vtkTransform::SafeDownCast(myViewTransform->GetInverse())); 

The executable freezes before showing the window.

I looked into the code and here is what happens:

  • when calling vtkCamera::SetUserViewTransform, the vtkCamera creates a vtkCameraCallbackCommand and adds it as an Observer of the userViewTransform
  • When any changes happen to the camera (e.g. SetPosition), the camera recomputes its internal transformation by composing userViewTransform with other transformations in vtkCamera::ComputeViewTransform()
  • This in turn calls userViewTransform->Update() through a GetMatrix() on the concatenated transform
  • In vtkAbstractTransform::Update(), UpdateMutex is locked
  • There is here a special case for inverse transforms:
// check to see if we are a special 'inverse' transform
  if (this->DependsOnInverse && this->MyInverse->GetMTime() >= this->UpdateTime.GetMTime())
  {
    vtkDebugMacro("Updating transformation from its inverse");
    this->InternalDeepCopy(this->MyInverse);
    this->Inverse();
    vtkDebugMacro("Calling InternalUpdate on the transformation");
    this->InternalUpdate();
  }
  • vtkTransform::Inverse does invert the matrix, but it also calls this->Modified()
  • Modified() evokes a ModifiedEvent
  • This calls the vtkCameraCallback defined earlier
  • In the Exec() method of the callback, it calls vtkCamera::ComputeViewTransform() again while we are still in it
  • We are in a loop, and we reach again the updateMutex.lock() which blocks forever.

Possible solution
The vtkCamera should disable its callback in the UserViewTransform when running ComputeViewTransform, (e.g. what is done in the destructor)

void vtkCamera::ComputeViewTransform()
{
  // main view through the camera
  this->Transform->Identity();
  if (this->UserViewTransform)
  {
    ***this->UserViewTransform->RemoveObserver(this->UserViewTransformCallbackCommand);
    this->Transform->Concatenate(this->UserViewTransform);
  }
  this->Transform->SetupCamera(this->Position, this->FocalPoint, this->ViewUp);
  this->ViewTransform->Identity();
  this->ViewTransform->Concatenate(this->Transform->GetMatrix());
  ***if (this->UserViewTransform){
  ***  this->UserViewTransform->AddObserver(
  ***        vtkCommand::ModifiedEvent, this->UserViewTransformCallbackCommand);
  ***}
}

However there might be other places where the camera updates its transform outside ComputeViewTransform that may trigger the same deadlock.

Any comment or idea ? How should I proceed to get this bug corrected in VTK ?

1 Like

Thanks for your detailed analysis. The official issue page for VTK is https://gitlab.kitware.com/vtk/vtk/issues, but here works too.

The main fault lies with the vtkTransform. A mutex lock around code that calls Modified(), or any method that can trigger callbacks, is guaranteed to eventually cause a deadlock in someone’s application. So rather than patching camera, I’m thinking of a patching the transform so that the Modified() call is deferred until the mutex is unlocked.

However, I must say that I was surprised to see that vtkCamera is observing a ModifiedEvent. We usually design our code to check the ModifiedTime of objects, rather than listen for ModifiedEvent.

Once I have a patch, I’ll report back here.

I’ve submitted a transform patch at gitlab.kitware.com/vtk/vtk/merge_requests/11574. It still has to go through review, so I can’t say exactly when it will be merged.

1 Like

Thank you very much ! I will follow the integration of the patch.

And yes, I was surprised too about the ModifiedEvent and the Observer.
I think this might be necessary because the userViewTransform might be updated by the Application without triggering a refresh of the camera view. As long as no one moves the camera or sets a Transform, I don’t think the camera will check for the ModifiedTime of the transforms.

But observing the ModifiedEvent of the transform won’t catch all changes to the transform, either. For instance, if A = B->GetInverse(), then changing B won’t cause A to generate a ModifiedEvent.

As an example of how ModifiedTime can be used properly, consider the UserTransform of vtkProp3D (the base class of vtkActor). For VTK actors, the ModifiedTime of the actor includes the ModifiedTime of the UserTransform, which in turn includes the ModifiedTime of all the other transforms that UserTransform depends upon. So when the vtkRenderer renders the scene, it considers the ModifiedTime of the actor’s transforms.

In comparison, the ModifiedTime of vtkCamera does not consider the ModifiedTime of the UserViewTransform. So it’s the responsibility of the application to call Modified() on the camera whenever the UserViewTransform changes. Trying to automate this with a transform -> camera callback is not only messy, but it doesn’t work in all situations (see first paragraph above).

I’d be in favor of changing the camera user transforms to work the same way as the actor user transforms, i.e. with ModifiedTime checks to ensure that everything is up-to-date when the render occurs.

Thanks for the explanation, especially how it works with Actors; seems that it would be a much better solution.