We have been working on this in 3D Slicer for many months and tried many possible approaches. We ended up using vtkGlyph3D/vtkGlyph2D filters for this, and use a separate pipeline (point source, glypher, mapper, actor) for selected, non-selected, and highlighted points. The performance is good for tend of thousands of points.
See a short demo on the video below of interacting with 1000 and 10000 labeled points; and find link to source code and some more information in this post.
