I use vtkPropPicker to get the current mouse position then I use vtkCellLocator to found the closest point from the picked position to the Polyline using FindClosestPoint method. This method works great only if the pick point is very close to the polyline. Once the mouse is not near the polyline, the closestPointMethod return the first element of my polyline.
I tried to use the method FindClosestPointWithinRadius to increase the distance between the mouse and the polyline but nothing changed.
I found the source of the error: the picking coordinates returned when I don’t pick any actor is not as what I expected.
I tried to use vtkWorldPointPicker like any other picker but I have those warnings:
2019-12-10 10:52:28.313 ( 22.811s) [ ] vtkOpenGLState.cxx:203 WARN| Error in cache state for GL_READ_FRAMEBUFFER_BINDING
2019-12-10 10:52:28.334 ( 22.832s) [ ] vtkOpenGLState.cxx:265 WARN| at stack loc
at vtksys::SystemInformationImplementation::GetProgramStack in ....
For information I use the master branch of VTK with Qt 5.12.5 on Windows 10 Pro and with a Nvidia 2080 Max-Q.
Since it is very easy to find closest point on a line and we want to allow some picking tolerance, we found that a fast and robust solution is to transform all the lines to display coordinates and find the closest line segment there:
Thank you for your answer! This solution works great and fast. I used vtkSelectVisiblePoints filter to select all visible points on the display and vtkInteractorObserver::ComputeDisplayToWorld and ComputeWorldToDisplay methods.
There is a limit of using only visible points: if no connexity verification between two consecutive points are done then the closest point on the segment can be in the “void”. For me it’s a very small limitation so I will not improve for now this algorithm.