Re-introducing a generic RenderingVR API for retrieving tracker state

This post summarizes offline discussions between @LucasGandel and @jcfr and is intended to help converge to having an API associated with the vtkRenderingVR module allowing to retrieve metadata associated with devices (aka trackers).


Text below adapted from offline answer by @LucasGandel

It should be possible to have a common interface for both OpenXR and OpenVR.

However, having a new API returning the OpenVR/OpenXR pose structure may not be the right approach.

Indeed, we already have 2 different places where we can query the vtkMatrix4x4 of a device:

  • from the event data in widgets/interactor styles,
  • or from vtkVRRenderWindow::GetDeviceToPhysicalMatrixForDeviceHandle.

Instead, the idea would be to add the missing information (DeviceTracked, DeviceConnected, PoseValid) to the generic vtkVRRenderWindow::DeviceData structure, and then provide generic accessors similar to the ones existing for the vtkMatrix4x4. For example, by adding something like:

bool vtkVRRenderWindow::IsDeviceConnectedForDeviceHandle(uint32_t handle);
bool vtkVRRenderWindow::IsDeviceTrackedForDeviceHandle(uint32_t handle);
bool vtkVRRenderWindow::IsPoseValidForDeviceHandle(uint32_t handle);

This way we would have a generic way to access all the required information without adding one more public function that returns the pose/matrix.


As part of the refactoring adding vtkRenderingVR now used in vtkRenderingOpenVR and newly introduced vtkRenderingOpenXR[Remoting] modules, the following API for retrieving pose metadata has been removed:

vr::TrackedDevicePose_t* vtkRenderingOpenVR::GetTrackedDevicePose(
  vtkEventDataDevice idx, uint32_t index)

and instead the generic API for retrieving the pose as a vtkMatrix4x4 was added:

vtkMatrix4x4* vtkVRRenderWindow::GetDeviceToPhysicalMatrixForDevice(vtkEventDataDevice idx)
vtkMatrix4x4* vtkVRRenderWindow::GetDeviceToPhysicalMatrixForDeviceHandle(uint32_t handle);

For reference,:

  • vtkRenderingOpenVR specific API was removed in 1aac3fe5f (Create VRInteractorStyle and VRRenderWindowInteractor)
  • vtkVRRenderWindow generic API for retrieving the pose was ultimately added in 9bd64d6 (Cleanup and rework of the VRInteractorStyle and subclasses)


In the context of the SlicerVirtualReality extension, retrieving the state of the tracker is relevant.

More specifically, we are interested in the following:

  • Based on eTrackingResult, we can detect if the controller is active. See here, here and here.
  • Based on bDeviceIsConnected, we can detect if the controller is
    connected. See here
  • We also retrieve the “status” of the pose based of bPoseIsValid and eTrackingResult. See here

To that effect, API specific to vtkRenderingOpenVR is been re-introduced in the Slicer/VTK fork as:

void vtkOpenVRRenderWindow::GetOpenVRPose(vtkEventDataDevice dev, vr::TrackedDevicePose_t** pose);
void vtkOpenVRRenderWindow::GetOpenVRPose(vtkEventDataDevice dev, uint32_t index, vr::TrackedDevicePose_t** pose);

See here

Details associated with vr::TrackedDevicePose_t struct

Mapping between OpenXR and OpenVR

Text below adapted from offline answer by @LucasGandel

The concept is a bit different with OpenXR, but in the end it should be able to provide the same information:

OpenVR OpenXR
bDeviceIsConnected isActive bool presented in section 11.5

@LucasGandel In addition of the state described above, we were also reporting the current status of the tracker by querying the TrackedDevicePose_t::TrackingResult, is there a similar concept offered by OpenXR ?

So far I could not find anything about calibration. But TrackingResult_Running_OK corresponds to the *_TRACKED_BIT and *_VALID_BIT flags being set, while TrackingResult_Running_OutOfRange should correspond to *_TRACKED_BIT flag being unset and the *_VALID_BIT flag still being set.