Thanks everyone for the quick replies and very useful information.
I managed to move points around on the mesh in an efficient way using the HardwareSelector (as mentioned by @Sebastien_Jourdain). I however didn’t find how to use de depth map of only the mesh. Currently, it’s using the complete renderWindow, so if a ball lies on top of the mesh, the HardwareSelector will return a wrong coordinate. Using the CellPicker solves this issue (since I can specify the actors to pick from, ignoring the actors for the spheres), but is indeed noticeably slower. Am I not seeing something, or is this indeed just a downside of using the HardwareSelector?
Small additional question: is there an easy way to do the inverse operation, getting the (2D) screen position of a 3D point on the mesh?
As for the math on the mesh, I’m able to do these calculations myself if needed.
@lassoan thanks for the suggestions. This software indeed looks very promising. I am however looking for a web-based approach and porting all algorithms to vtk.js isn’t feasible at this point.