Problems with multi-touch gestures with current sources

Hello everyone,

I built the latest VTK sources from repository (VTK master, VS2017, x64, Qt 5.9.1, Windows 10 Pro), built Examples/GUI/Qt/TouchGestureViewer example and tried it on several machines. I got same problems on all machines (Surface Pro 6 and Windows 10 laptop with multi-touch touch-pad):

When I rotate using 1 finger, I get very nice and smooth rotation, but, when I pan / pinch using 2 fingers, I get jerky motion with performance downgrade with at least 5x factor.

I noticed this problem is solved in Slicer3D application, but, I couldn’t decode what is changed in interactor’s sources. Can someone help with how to reduce this jerkiness in TouchGestureViewer please? I’m adding multi-touch gestures support to my VTK based software and this jerkiness is really bothering me :sneezing_face:

Thanks,
Nenad.

When we fixed multi-touch interactions in 3D Slicer, we contributed all necessary changes to VTK (and created the TouchGestureViewer for testing in a pure VTK environment), so you should be able to make this work.

For rendering simple scenes, the interaction should be fluid, as shown in the example video in the pull request. If you want to render a more complex scene, then you probably need to reduce number of re-renders (because hundreds of gesture events may be created per second). In 3D Slicer, we use CTK’s ctkVTKAbstractView as VTK render widget and use custom interactors that do not call Render() directly but use requestRender(), which ensures that rendering takes place at the specified MaximumUpdateRate.

If you develop medical applications then I would suggest to use 3D Slicer as a basis (implement new applications by customizing/extending 3D Slicer), as we have already solved hundreds of issues like this - that are important for a full-featured end-user application but has not been or cannot be implemented at VTK level. Similarly, for engineering/technical computing domain, you may consider using ParaView as a basis of your application.

Andras,

Thank you very much for your help! Unfortunately, I still have jerky motions on our multi-touch devices. I tried to build TouchGestureViewer example with following versions of VTK sources - 31dc6a08b8268133eb8bad83b7a65d70535673fa (VTK version used by the Slicer’s latest release) and 6488ef5d38d2eae1ff79d78a0922a7f5b6af828f (VTK sources just after multi-touch gestures were added) and we still get jerky motions when performing panning and pinching :frowning:

I used VS2017, x64, with default options for building VTK (I only included building Qt support modules) and Qt v5.14.0.

Could I ask you to send us package with built TouchGestureViewer which works like on your video, smoothly, so we could test it on our side, please? Perhaps the problem is on our machines / environments?

Thanks in advance,
Nenad.

@Sunderlandkyl Could you upload a TouchGestureViewer executable and share the link? Do you have a guess of why the behavior could be different between TouchGestureViewer and Slicer?

Not sure.
I’m building the latest version of VTK and will do some tests + add a link when it’s compiled.

Here is the executable:

I see what you mean about performance issue.
Debugging hasn’t revealed any cause yet. Despite the apparent slowdown, there is no corresponding spike in CPU usage. I’ll keep looking into this.

We’ve done some more testing.

The performance was much better when we switched from integrated to dedicated graphics, and was likely what was used for the demo video in the pull request.
It was also much faster when we changed the the high dpi scaling settings to system (at the cost of resolution).

It’s still not obvious what the difference is that makes Slicer faster than the example application. It may be a congestion of events from the gestures that are causing the performance issue.

3D Slicer uses “requestRender” mechanism (implemented in CTK) to ignore too frequent re-rendering requests. This mechanism may prevent flooding of the message queue with GUI events.