Hi, I’m trying to get 2D coordinates of point labels visualized based on the code from the example https://examples.vtk.org/site/Cxx/Visualization/LabelPlacementMapper/. I want to allow the users to interact with the labels, so my goal is to get the bounds of visible (not obscured) labels.
I assume that each separate label has a corresponding vtkProp/vtkActor storing information about 2D position that is used to render them on the screen. I can’t find a way to access the labels though. I tried using 2D prop picker, but it didn’t return anything.
Would you please advise? Is it possible to get such information from vtkLabelHierarchy or vtkLabelPlacer?
The thing is I’m not interested in the points (represented by spheres), but rather in labels that are being displayed next to them as boxes containing text. Ideally, I would like to click on such label to get an underlying actor or get the coordinates of all the visible labels.
Picking the points is definitely not an option. I worked with vtkButtonWidget before, which can be clicked and will highlight when hovering over it with the mouse. Using the second option for placement on the vtkButtonWidgetRepresentation, you may get what you want. I remember that I used it once, but had to back a callback on the interactor to adjust the normals of the placement. Perhaps using vtkProp3DButtonRepresentation can be used, but then I believe you need to define props as boxes containing text
I don’t think I will be able to get any existing callbacks to work with the labels. My biggest problem is to even find how these labels are being generated and mapped to 2D screen coordinates. Maybe then I could use ray-casting to check which label box was clicked. I’m interested in the whole bounding box marked red below (not the points themselves).
I think this is a bad approach. Did you try using vtkButtonWidget? There is an example with an elliptical button. https://examples.vtk.org/site/Cxx/Interaction/EllipticalButton/ You basically just need to replace the ellipsis with a box with a text label and associate actors to that. Do you use pickable labels in VMTK@lassoan ?
We spent about 10 years and $500k to make VTK widgets work in Slicer, but everything was too complicated yet too limited. We could only achieve what we needed (including all what you need - showing labels only for visible points, interact with points, etc.) by keeping only the widget and representation base classes and redevelop everything else on top of them. We did lots of application-specific optimizations (e.g., using 3D Slicer data objects directly in the widget and representation) so that we could not contribute these back to VTK.
Overall, developing interactive 3D widgets is so complicated that if you need such widgets then I would recommend to customize an existing application that already have them, such as 3D Slicer or ParaView.
I’m using Qt in Python as a framework for UI, so I will probably not be using any VTK widgets directly. That’s why my question focused on the underlying mechanisms of label placement to understand it better
The whole thing seems quite complicated under the hood, so I wanted to get a grasp of how the positions of labels are being calculated internally. Labels are fully interactive, they respond to 3D meshes and each other by being obscured etc, so I suspect that there is an algorithm that calculates the 3D relations between the labels. There are probably a couple of steps, but I’m looking for something that accepts labels in 3D and outputs filtered 2D coordinates that will be displayed on the screen.
That’s way I wanted to understand where the actual 2D props of the rendered labels are stored at the moment of calculation (probably somewhere in vtkLabelHierarchy/vtkLabelPlacer?), but I couldn’t find any such objects.
I did that a while back using vtkButtonWidgets as I told you. It is a little complicated since you need to do text rendering and make and make some complicated observers to the interactor. It is a little simpler in C++ since there is a number of classes that you cannot inherit from. I invited @lassoan into the talk, because he provided me with this link, which you need for passing userdata in your callback to python. Documentation/Nightly/Developers/FAQ - Slicer Wiki.
To be honest, I would use Slicer, MITK or ParaView as @lassoan suggested. It is a lot simpler. Give it a try. If you create a simple example and post some code, I could possible help you out. I don’t have a good example that you can use.
