The hardware selector can be used to create a depth map of just your mesh so you could do displayToWorld the way @Forrest is explaining but using a cached buffer of your rendered geometry. This way such operation could be very fast and would not require any complex math that might not be available in JS.