A few months ago I found a very small issue with the XRenderWindowInteractor where I could not trigger
? keysym on my program.
This led me to a rabbit hole and an issue that I’m currently fixing here: https://gitlab.kitware.com/vtk/vtk/-/merge_requests/10652?
However, in the process, I’ve leaned a lot about how key interactions are handled and especially how VTK is limited here. I of course am talking about pure VTK, one can always rely on another framework like Qt to handle key interaction.
First some vocabulary.
- Virtual key: A OS dependant value corresponding to a physical key on the keyboard. Not impacted by keyboard layout, not impacted by modifiers but non consistent accross OS.
- KeyCode: A single (unsigned, but read below) char on the extended ASCII table corresponding to a Key being pressed, impacted by BOTH modifiers and layout.
- KeySym: A string representation of a key being pressed, eg
BaskSpacebut can be a single char for letters. Impacted by both modifiers and layout.
- KeyCode and KeySym are inspired by Xorg API, but KeyCode is a cross platform concept (although under different names). KeySym doesn’t really exist accross OS but VTK tries to figure it out anyway.
Here is how it currently works.
- A physical key is pressed and the OS receive it, send the information to VTK with its Virtual Key.
- VTK receive the event and the Virtual Key
- VTK ask the OS to convert the Virtual Key to a KeyCode, then uses different methods to produce a KeySym from the KeyCode
One would say it is all well and good, however, I conducted some testing, here are the results (QWERTY Keyboard):
The first striking thing is that
Control modifier has impact on the KeyCode, which could have been unexpected until you learn about, well,
Control Codes: Control codes - converting to ASCII, hex or decimal
Indeed, control codes is a cross platfrom ancient behavior, but they do impact the KeyCode.
But then we can see the outside of control codes (eg: Ctrl + 1), Win32 behavior is different.
Here is the most important question of this post:
As a VTK user, do you expect
GetKeyCode to provide Control codes or not ?
In any case, we definitely need to add a
GetKeyCodeWithouModifiersExceptShift method somewhere. Turns out MacOS already thought of that and provide it in its API, so I think either modifying GetKeyCode in that direction or adding a new method makes complete sense.
We also need to ensure keycode consistency across OS when pressing Control.
Without it, we can ONLY rely on keysym.
- KeyCode consistency
Outside of the
Control modifier, we can see that certain key do not behave consistently in regards to KeyCode, this should be fixed. to be fixed.
- KeySym consistency
For certain characters, KeySym is not consistent, this is because outside of XOrg, the KeyCode → KeySym mapping is done in tables inside VTK and these tables are just not complete, this should be fixed. to be fixed.
MacOS has multiple issues, for starters
Alt modifier is completely broken. Moreover, it should default KeySym to
None instead of nullptr. to be fixed.
- Extended ASCII
Outside of QWERTY there are other keyboard layout wich have keys in the extended ASCII table. It works surprisingly well for the keycode (Win32 and Xorg) however GetKeyCode API uses
unsigned char as it should. Issue to be open.
- vtkCallBackMapper:: SetCallbackMethod
This API is very confusing:
void vtkWidgetCallbackMapper::SetCallbackMethod ( unsigned long VTKEvent, int modifiers, char keyCode, int repeatCount, const char * keySym, unsigned long widgetEvent, vtkAbstractWidget * w, CallbackType f )
Because it support
vtkEvent::AnyModifiers, yes the keyCode and the keySym and impacted by the modifier, so if a developer want a callback to be called whatever the modifier, the following is required:
this->CallbackMapper->SetCallbackMethod(vtkCommand::KeyPressEvent, vtkEvent::AnyModifier, 'p', 1, "p", vtkWidgetEvent::PickPoint, this, vtkDisplaySizedImplicitPlaneWidget::PickOriginAction); this->CallbackMapper->SetCallbackMethod(vtkCommand::KeyPressEvent, vtkEvent::AnyModifier, 'P', 1, "P", vtkWidgetEvent::PickPoint, this, vtkDisplaySizedImplicitPlaneWidget::PickOriginAction); this->CallbackMapper->SetCallbackMethod(vtkCommand::KeyPressEvent, vtkEvent::AnyModifier, 16, 1, "p", vtkWidgetEvent::PickPoint, this, vtkDisplaySizedImplicitPlaneWidget::PickOriginAction);
Very unpractical. One could imagine a better API where only a case agnostic KeySym is provided.
Issue to be open.
does not behave as developpers expect, should we change its behavior to behave as expected or provide new, clearer API to do so, eg:
unsigned char vtkRenderWindowInteractor::GetKeyCodeWithoutModifiers() and
unsigned char vtkRenderWindowInteractor::GetKeyCodeWithoutModifiersExceptShift()