OpenVR problems with VTK 9.2

VR/AR
9

Yes it works in both VTK 9.1 and 9.2, and looks (on a quick glance) the same:

image

10

Well, someone needs to go digging to figure out what is going on.

Maybe @LucasGandel has an idea.

11

Looks like this fix is not part of the release.
@codeling Could you please cherry-pick the above commit to see if it fixes the issue?

2 Likes
12

Yes this seems to solve the volume rendering issue. Thanks!

Not sure if it solved my original issue (left eye rendering only) - I’ll check back with our main program…
EDIT: this specific fix unfortunately not. I’ll have to keep looking for a minimal reproducible example (will try with git master next though)

13

Thanks a lot for the feedback. Do I get it correctly that both eyes are now rendered correctly in the small example you shared above, but not in your main application?

In your main program, do you use any of the following methods?

14

Correct.

In your main program, do you use any of the following methods?

Not as far as I (/grep) can see at the moment.
These would be known for causing such issues?

15

Thanks a lot for checking.
I experienced a similar issue where only the left eye was rendered, depending on where the prop was placed in the scene, and I thought this was caused by the physical scale of the render window but this might just be a side-effect of the bug.

IIRC the problem was that the props were always culled because the clipping range of the vtkCamera was wrong for the right eye. Can you try adding the following to your main application to see if it fixes the issue?

renderer->RemoveCuller(renderer->GetCullers()->GetLastItem());

2 Likes
16

Yes, this fixes the issue with the right eye!

It doesn’t seem to be the only culling problem though:

For volume rendering, we still see some issue - experimenting with a volume (grayscale fiber rendering), it starts to show as some kind of “white fog”; by interactions I can make the volume smaller so it becomes visible as a whole; but when the volume is close enough to be cut by the the near plane, the cut doesn’t seem to happen in direction of the viewer:

image
image1042×649 101 KB

I guess this is a similar or related culling issue?

EDIT: how it looks when starting is depending on the spacing of the volume (only the rendering, not the size). With sufficiently small spacings (e.g. 0.05), I don’t see the “white fog” (which appeared with spacing > 0.5) but the volume gets more or less rendered (though already then, being cut int the “wrong direction”):

image
image852×506 67.5 KB

(we see the front of the volume - you see the bounding box in the upper left, even that is “cut” towards the back it seems).
That the bounding box is cut is something happening only in the “initial” view - if I move the volume around, the box starts to stay visible as a whole, only the volume gets wrongly culled.

EDIT2: This volume rendering behavior can actually be reproduced by my example code above - look how you can “see through” the volume when you are actually supposed to be “in it”:

image
image1048×643 35.6 KB

17

Thanks for the feedback, this is great news that the issue with the right eye is solved.
From your report, I identify the following issues:

  1. Small spacing required to remove white fog
    The problem here is probably the default value of the vtkVolumeProperty::ScalarOpacityUnitDistance parameter being too big. Can you try calling volumeProperty->SetScalarOpacityUnitDistance(0.001); ?

  2. The bounding box is cut
    This could definitely be related to 1. or to the wrong camera clipping range. Does calling renderer->ResetCameraClippingRange() help?

  3. Volume is wrongly cut by the camera near plane
    I confirm this is an issue. I have experienced this in the past but unfortunately I don’t have a fix for it yet.
    The problem is again related to the camera clipping range. The GPU volume mapper should add a clipping plane just in front of the camera near plane, in order to “close” the hole that appears in your first screenshot (see here).
    I suspect that the plane is wrongly placed because the physical scale or something is not taken into account.

1 Like
18

Thanks for the lightning fast reply :smiley:

this is great news that the issue with the right eye is solved.

Do you have an idea how a more permanent fix of this might look like?

Regarding the volume rendering points:

  1. SetScalarOpacityUnitDistance doesn’t help, it was caused by similar reasons than 2:
  2. Yes it helped! We actually had that already in the code, but I had experimented with moving VR initialization (including ResetCamera, ResetCameraClipping and event loop start) to before adding the rendering stuff. When calling these after adding the renderers, it works again.
  3. Looking forward to a fix, don’t think I can help here unfortunately.
19

TBH I have no idea, the code was there in tests since the 1st commit that introduced OpenVR support in VTK (see here).
Maybe using vtkCameraActor/vtkFrustumSource could help to confirm that the camera frustum planes are computed correctly for the VR camera.

1 Like
20

What I’m wondering here is then why this was working for us without the workaround of removing the last culler for VTK < 9.2…
Also the volume cutting problem is new - with VTK 9.1 it worked as expected.

Maybe using vtkCameraActor/vtkFrustumSource could help to confirm that the camera frustum planes are computed correctly for the VR camera.

If I have time I might look into it, thanks for the pointers!

21

Just tested, the last problems with wrong clipping seem to be fixed in VTK 9.3 (latest release candidate rc2)!

The above example program runs nicely, except for a few errors in the console:

vtkOpenGLFramebufferObj:356    ERR| vtkOpenGLFramebufferObject (000001419E3E3D40): failed at glDeleteFramebuffers 16 OpenGL errors detected

Edit - Correction: The near plane clipping problem is not yet fully fixed, but it has improved somewhat - the volume in the example code above is shown correct until the HMD gets close to the volume borders. When very close or in the volume, the clipping plane is quite far away from the eyes (nearly on the opposite side of the volume, but at least not in the wrong direction anymore).

22

@codeling Thanks a lot for the feedback. I think this is the corresponding VTK issue : https://gitlab.kitware.com/vtk/vtk/-/issues/19123.
There I proposed a fix, but did not open any MR as I thought something cleaner could be done. However, this might be better than the current bug for now if you want to open a MR for it, I’ll be happy to review.

23

Just tested with git master (specifically, ff92d0720f), and the near clipping plane issues that I described above are gone (probably due to https://gitlab.kitware.com/vtk/vtk/-/merge_requests/10783 having been merged?).

1 Like
24

I confirm the camera near plane clipping issue was fixed in VTK!10783. It might also help with the RemoveCuller workaround, I would be curious to see if the “right eye” issue is also fixed by this MR.

25

Your intuition is right - the workaround is not required anymore!

1 Like
26

This is an awesome news! Thank you so much for testing, this is very helpful!

27

Does anybody know whether the merge mentioned above has any chance of making it into the next VTK release (probably 9.3.1, right?)? I tried to check in git but am not sure how to determine this… is there an easy way to determine the commit with which the request was merged? and what is the branch that will turn into version 9.3.1 - probably “release”?

28

Yes: https://gitlab.kitware.com/vtk/vtk/-/merge_requests/10854

and what is the branch that will turn into version 9.3.1 - probably “release”?

Yes, if a 9.3.1 is released.

1 Like