# Re-introducing a generic RenderingVR API for retrieving tracker state
**URL:** https://discourse.vtk.org/t/re-introducing-a-generic-renderingvr-api-for-retrieving-tracker-state/10394
**Category:** VR/AR
**Created:** [January 5, 2023, 2:28pm UTC](https://discourse.vtk.org/t/re-introducing-a-generic-renderingvr-api-for-retrieving-tracker-state/10394 "2023-01-05T14:28:56Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@jcfr](https://discourse.vtk.org/u/jcfr)
#### Post date: [January 5, 2023, 2:28pm UTC](https://discourse.vtk.org/t/re-introducing-a-generic-renderingvr-api-for-retrieving-tracker-state/10394/1 "2023-01-05T14:28:56Z")
This post summarizes offline discussions between @LucasGandel and @jcfr and is intended to help converge to having an API associated with the `vtkRenderingVR` module allowing to **retrieve metadata associated with devices** (aka trackers).
### Proposal
_Text below adapted from offline answer by @LucasGandel_
It should be possible to have a common interface for both `OpenXR` and `OpenVR`.
However, having a new API returning the OpenVR/OpenXR pose structure may not be the right approach.
Indeed, we already have 2 different places where we can query the `vtkMatrix4x4` of a device:
- from the event data in widgets/interactor styles,
- or from `vtkVRRenderWindow::GetDeviceToPhysicalMatrixForDeviceHandle`.
Instead, the idea would be to add the missing information (DeviceTracked, DeviceConnected, PoseValid) to the generic `vtkVRRenderWindow::DeviceData` structure, and then provide generic accessors similar to the ones existing for the `vtkMatrix4x4`. For example, by adding something like:
```cpp
bool vtkVRRenderWindow::IsDeviceConnectedForDeviceHandle(uint32_t handle);
bool vtkVRRenderWindow::IsDeviceTrackedForDeviceHandle(uint32_t handle);
bool vtkVRRenderWindow::IsPoseValidForDeviceHandle(uint32_t handle);
```
This way we would have a generic way to access all the required information without adding one more public function that returns the pose/matrix.
### Background
As part of the refactoring adding `vtkRenderingVR` now used in `vtkRenderingOpenVR` and newly introduced `vtkRenderingOpenXR[Remoting]` modules, the following API for retrieving pose metadata has been removed:
```cpp
vr::TrackedDevicePose_t* vtkRenderingOpenVR::GetTrackedDevicePose(
vtkEventDataDevice idx, uint32_t index)
```
and instead the generic API for retrieving the pose as a `vtkMatrix4x4` was added:
```cpp
vtkMatrix4x4* vtkVRRenderWindow::GetDeviceToPhysicalMatrixForDevice(vtkEventDataDevice idx)
vtkMatrix4x4* vtkVRRenderWindow::GetDeviceToPhysicalMatrixForDeviceHandle(uint32_t handle);
```
For reference,:
- `vtkRenderingOpenVR` specific API was removed in [1aac3fe5f](https://github.com/Slicer/VTK/commit/1aac3fe5f) (`Create VRInteractorStyle and VRRenderWindowInteractor`)
- `vtkVRRenderWindow` generic API for retrieving the pose was ultimately added in [9bd64d6](https://github.com/Slicer/VTK/commit/9bd64d6) (`Cleanup and rework of the VRInteractorStyle and subclasses`)
### Workaround
In the context of the [SlicerVirtualReality](https://github.com/KitwareMedical/SlicerVirtualReality#readme) extension, retrieving the state of the tracker is relevant.
More specifically, we are interested in the following:
- Based on `eTrackingResult`, we can detect if the controller is active. See [here](https://github.com/KitwareMedical/SlicerVirtualReality/blob/f37331ee4169b3a940705a7570720e4c48d3428d/VirtualReality/Widgets/qMRMLVirtualRealityView.cxx#L537-L544), [here](https://github.com/KitwareMedical/SlicerVirtualReality/blob/f37331ee4169b3a940705a7570720e4c48d3428d/VirtualReality/Widgets/qMRMLVirtualRealityView.cxx#L634-L641) and [here](https://github.com/KitwareMedical/SlicerVirtualReality/blob/f37331ee4169b3a940705a7570720e4c48d3428d/VirtualReality/Widgets/qMRMLVirtualRealityView.cxx#L578-L592).
- Based on `bDeviceIsConnected`, we can detect if the controller is
connected. See [here](https://github.com/KitwareMedical/SlicerVirtualReality/blob/f37331ee4169b3a940705a7570720e4c48d3428d/VirtualReality/Widgets/qMRMLVirtualRealityView.cxx#L546-L553)
- We also retrieve the “status” of the pose based of `bPoseIsValid` and `eTrackingResult`. See [here](https://github.com/KitwareMedical/SlicerVirtualReality/blob/f37331ee4169b3a940705a7570720e4c48d3428d/VirtualReality/Widgets/qMRMLVirtualRealityView.cxx#L673-L674)
To that effect, API specific to `vtkRenderingOpenVR` is been re-introduced in the `Slicer/VTK` fork as:
```auto
void vtkOpenVRRenderWindow::GetOpenVRPose(vtkEventDataDevice dev, vr::TrackedDevicePose_t** pose);
void vtkOpenVRRenderWindow::GetOpenVRPose(vtkEventDataDevice dev, uint32_t index, vr::TrackedDevicePose_t** pose);
```
See [here](https://github.com/Slicer/VTK/compare/97a187572d...Slicer:VTK:slicer-v9.1.20220125-efbe2afc2)
### Details associated with `vr::TrackedDevicePose_t` struct
>
### Mapping between OpenXR and OpenVR
_Text below adapted from offline answer by @LucasGandel_
The concept is a bit different with OpenXR, but in the end it should be able to provide the same information:
| OpenVR | OpenXR |
| --- | --- |
| `eTrackingResult` | `XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT` and `XR_SPACE_LOCATION_POSITION_TRACKED_BIT` flags presented in [section 7.4.1](https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#_xrlocatespace) |
| `bDeviceIsConnected` | `isActive` bool presented in [section 11.5](https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#_reading_input_action_state) |
| `bPoseIsValid` | `XR_SPACE_LOCATION_ORIENTATION_VALID_BIT` and `XR_SPACE_LOCATION_POSITION_VALID_BIT` flags presented in [section 7.4.1](https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#_xrlocatespace) |
---
### Author:  [@jcfr](https://discourse.vtk.org/u/jcfr)
#### Post date: [January 5, 2023, 2:47pm UTC](https://discourse.vtk.org/t/re-introducing-a-generic-renderingvr-api-for-retrieving-tracker-state/10394/2 "2023-01-05T14:47:48Z")
@LucasGandel In addition of the state described above, we were also reporting the current status of the tracker by querying the `TrackedDevicePose_t::TrackingResult`, is there a similar concept offered by OpenXR ?
>
---
### Author:  [@LucasGandel](https://discourse.vtk.org/u/LucasGandel)
#### Post date: [January 6, 2023, 7:48am UTC](https://discourse.vtk.org/t/re-introducing-a-generic-renderingvr-api-for-retrieving-tracker-state/10394/3 "2023-01-06T07:48:07Z")
So far I could not find anything about calibration. But `TrackingResult_Running_OK` corresponds to the `*_TRACKED_BIT` and `*_VALID_BIT` flags being set, while `TrackingResult_Running_OutOfRange` should correspond to `*_TRACKED_BIT` flag being unset and the `*_VALID_BIT` flag still being set.