About KeyCode, KeySym, Modifiers, Crossplatform and reliable code

A few months ago I found a very small issue with the XRenderWindowInteractor where I could not trigger ? keysym on my program.

This led me to a rabbit hole and an issue that I’m currently fixing here: https://gitlab.kitware.com/vtk/vtk/-/merge_requests/10652?

However, in the process, I’ve leaned a lot about how key interactions are handled and especially how VTK is limited here. I of course am talking about pure VTK, one can always rely on another framework like Qt to handle key interaction.

First some vocabulary.

  • Virtual key: A OS dependant value corresponding to a physical key on the keyboard. Not impacted by keyboard layout, not impacted by modifiers but non consistent accross OS.
  • KeyCode: A single (unsigned, but read below) char on the extended ASCII table corresponding to a Key being pressed, impacted by BOTH modifiers and layout.
  • KeySym: A string representation of a key being pressed, eg delete or BaskSpace but can be a single char for letters. Impacted by both modifiers and layout.
  • KeyCode and KeySym are inspired by Xorg API, but KeyCode is a cross platform concept (although under different names). KeySym doesn’t really exist accross OS but VTK tries to figure it out anyway.

Here is how it currently works.

  1. A physical key is pressed and the OS receive it, send the information to VTK with its Virtual Key.
  2. VTK receive the event and the Virtual Key
  3. VTK ask the OS to convert the Virtual Key to a KeyCode, then uses different methods to produce a KeySym from the KeyCode

One would say it is all well and good, however, I conducted some testing, here are the results (QWERTY Keyboard):

  1. Control modifier

The first striking thing is that Control modifier has impact on the KeyCode, which could have been unexpected until you learn about, well, Control Codes: Control codes - converting to ASCII, hex or decimal

Indeed, control codes is a cross platfrom ancient behavior, but they do impact the KeyCode.

But then we can see the outside of control codes (eg: Ctrl + 1), Win32 behavior is different.

Here is the most important question of this post:

As a VTK user, do you expect GetKeyCode to provide Control codes or not ?

In any case, we definitely need to add a GetKeyCodeWithouModifiersExceptShift method somewhere. Turns out MacOS already thought of that and provide it in its API, so I think either modifying GetKeyCode in that direction or adding a new method makes complete sense.

We also need to ensure keycode consistency across OS when pressing Control.

Without it, we can ONLY rely on keysym.

  1. KeyCode consistency

Outside of the Control modifier, we can see that certain key do not behave consistently in regards to KeyCode, this should be fixed. to be fixed.

  1. KeySym consistency

For certain characters, KeySym is not consistent, this is because outside of XOrg, the KeyCode → KeySym mapping is done in tables inside VTK and these tables are just not complete, this should be fixed. to be fixed.

  1. MacOS

MacOS has multiple issues, for starters Alt modifier is completely broken. Moreover, it should default KeySym to None instead of nullptr. to be fixed.

  1. Extended ASCII

Outside of QWERTY there are other keyboard layout wich have keys in the extended ASCII table. It works surprisingly well for the keycode (Win32 and Xorg) however GetKeyCode API uses char, not unsigned char as it should. Issue to be open.

  1. vtkCallBackMapper:: SetCallbackMethod


This API is very confusing:

void vtkWidgetCallbackMapper::SetCallbackMethod 	( 	unsigned long  	VTKEvent,
		int  	modifiers,
		char  	keyCode,
		int  	repeatCount,
		const char *  	keySym,
		unsigned long  	widgetEvent,
		vtkAbstractWidget *  	w,
		CallbackType  	f 

Because it support vtkEvent::AnyModifiers, yes the keyCode and the keySym and impacted by the modifier, so if a developer want a callback to be called whatever the modifier, the following is required:

  this->CallbackMapper->SetCallbackMethod(vtkCommand::KeyPressEvent, vtkEvent::AnyModifier, 'p', 1,
    "p", vtkWidgetEvent::PickPoint, this, vtkDisplaySizedImplicitPlaneWidget::PickOriginAction);
  this->CallbackMapper->SetCallbackMethod(vtkCommand::KeyPressEvent, vtkEvent::AnyModifier, 'P', 1,
    "P", vtkWidgetEvent::PickPoint, this, vtkDisplaySizedImplicitPlaneWidget::PickOriginAction);
  this->CallbackMapper->SetCallbackMethod(vtkCommand::KeyPressEvent, vtkEvent::AnyModifier, 16, 1,
    "p", vtkWidgetEvent::PickPoint, this, vtkDisplaySizedImplicitPlaneWidget::PickOriginAction);

Very unpractical. One could imagine a better API where only a case agnostic KeySym is provided.
Issue to be open.


char vtkRenderWindowInteractor::GetKeyCode
does not behave as developpers expect, should we change its behavior to behave as expected or provide new, clearer API to do so, eg:
unsigned char vtkRenderWindowInteractor::GetKeyCodeWithoutModifiers() and unsigned char vtkRenderWindowInteractor::GetKeyCodeWithoutModifiersExceptShift()

That is really nice that you pick this up. I have been using the vtkWidgetCallbackMapper in C++, which I find quite confusing. I often prefer to use VTK without Qt and even developed my own small library for executing a call in the main thread since I need an alternative to QueuedConnection. In Python, I sometimes ask for the ‘SHIFT’ modified in a callback and even if I hold down shift I often see a result that the modifier is not enabled. Here it would be much simpler if the keysym contained the modifier as well. Thanks for looking into this

Historically keysym has not contained the modifier, so making that change would break a lot of existing code.

VTK has a Qt-to-keysym translation table in the QVTKInteractorAdapter class, and we can possibly use this as our guide to “correct” behavior. This is what Paraview, Slicer, and other Qt/VTK applications use for setting the keysym in the interactor, and it would be nice if the native X11, Win32, macOS interactors give the same key information as e.g. Paraview’s interactor. But I’m not 100% sure if it handles control modifiers correctly, so it might also need fixes.

Indeed, I agree with @dgobbi here, the KeySym will not contain the modifier. The modifiers are accessible via the dedicated methods.

it would be nice if the native X11, Win32, macOS interactors give the same key information as e.g. Paraview’s interactor. But I’m not 100% sure if it handles control modifiers correctly, so it might also need fixes.

That is exactlyt what I’m aiming for.

Windows inconsistencies are being fixed here:

MacOS inconsistencies are being fixed here:

Just a quick follow up about Xorg implementation. It looks like Alt Gr (see ISO layout) does not set the Alt modifier on the RenderWindowInteractor. Not sure if this is the expected. The keysym is still good.

Quick follow up the MacOS implementation. Pressing the Option key is mapped to the Alt modifier, however the macos keyboard layout is not standard and pressing such key provides special KeyCode symbols.

I’ve update the document with the last code changes that are soon to be merged, I’ve extended my tests to include azerty layout too.

First, it is already looking much better, with more consistent keycodes and keysyms accross OSes. I remarked a few more things:

  1. Testing and keysyms on Win/MacOS

KeySym on windows and MacOS are not provided by the OS but builtin VTK based on the different OS provided keycodes. This explains why quoteright which is not even part of the Latin1 ASCII is listed there. It is a very easy fix, which I will take care of.

However it raises the question of testing. Currently the logic the translate a OS virtual key code into an actual keyCode and keySym is completely untested in VTK. Indeed, VTK testing system interface itself just after that.

I will investigate the possiblity of adding simple or even exhaustive testing of this layer.

  1. ASCII vs Latin1 support

Xorg implementation always nativaly supported Latin1 (ISO/IEC 8859-1 - Wikipedia) for KeyCode and KeySym, while Win32 supported it for KeyCode only and macOS did not support it at all. I’ve added support for KeyCode in MacOS as it was trivial but in order to support KeySym too, more work is needed.

Indeed, there are big translation tables from KeyCode to KeySym in MacOS and Win32. I will try to complete them but the process may prove hard, especially without testing.

Also, the value of these >=128 Latin1 keycode are currently negative as the API is using a char. This must change.

  1. Cross platform interaction

While Windows and Linux are quite respectful of the keyboard layout standard, MacOS does what MacOS do with custom keyboard and such, so the modifiers and layouts are slightly different.

Which means that certains combination of keys using Alt modifier will just not be available. You won’t be able to get Alt modifier with a keycode, because Alt + a on a MacOS is Å.

In a way, this is expected and visible on the physical key itself. If the user pressed the same combination of key in a text editor, the will get the Å as well!

But it also means that, when coding a cross platform widget in VTK, you cannot expect your interactions to work on all layout and all OSes.

One solution is to provide full configuration of the interaction. While this is usually possible at application level, this is not really an option in VTK Widgets.

This is where the API I suggested above start to make complete sense. For cross platform widgets, one absolutaly needs keycodes without modifiers. It is not bullet proof but way more stable accross layout and OSes. Basically the whole alphabet [a-z] would become fully cross-layout and cross-platform.

So I now believe this API addition is indeed required.

It’s good to see the better consistency (and to see that keysym is never nullptr).

The keysym quoteright just means ASCII singe quote, but that name has been deprecated in favor of apostrophe. The deprecated keysym name got into VTK’s char-to-keysym translation tables via Tk, which uses the obsolete name. From the X11/keysymdef.h header file in the Latin1 section:

#define XK_apostrophe          0x027
#define XK_quoteright          0x027	/* deprecated */
#define XK_grave               0x060
#define XK_quoteleft           0x060	/* deprecated */

The curly quotes have keysyms leftsinglequotemark and rightsinglequotemark (but I think no keyboards have these keys).

This is especially troublesome because char is not always signed. Returning any value outside of the range [0,127] is platform-dependent. There is also a difficulty when people pass the returned value to isprint() or similar… if the code is negative or out-of-range for the locale, then isprint() has undefined behavior (on MSVC, I have seen calls to isprint() do an abort() and exit if the value is out-of-range).

One real difficulty is that people expect GetKeyCode() to do two different things:

  1. identify the key that was pressed (identified by the letter/symbol/word printed on the key)
  2. return a string containing the text that is generated by the keypress including the effect of modifiers (possibly 0 chars of text or even multiple chars of text)

One single API can’t do both of these things. A proper fix would require GetKeyCode() to be split into two APIs. However, this small change would require quite a lot of work.

About quoteright:

Good point, for consistency, it still should be updated though.

About char and unsigned char:

I had no idea, I would expect a char to always be signed ? In any case, this is one more argument to use the correct api. In the meantime, I could improve GetKeyCode doc and provide an example of converting to unsigned char correctly before usage.

About GetKeyCode:

I don’t completely agree. I think GetKeyCode should take modifiers into account and provide the correct Latin1 ASCII encoded char corresponding to the Key pressed with the modifiers. This is how it was implemented and how it should be used. But I’m advocating to add more methods to provided indeed a GetKeyCodeIgnoringModifiers. I’m not enterely sure how it should be implemented but It should not be too hard to do.

Proper internal Latin1 support being added here: https://gitlab.kitware.com/vtk/vtk/-/merge_requests/10680

With the latest fix, I’ve updated my spreadsheet:

None of the remaining difference can be considered bugs. This close my (initial) work on this.
The odd behavior on MacOS with alt is expected, is that is the key symbol that is supposed to be emitted.


Working with Alt modifier is not recommended, depending on the OS in use, it will behave wildly different.

Working with GetKeyCode and modifiers is not recommended at all. It will not behave as you expect it.

Working with GetKeySym and modifiers work (outside of Alt) as long as one understands that Shift modifier may have an impact of the KeySym.

In the context of alphas (letters), using ToUpper on the KeySym with Shift and Control modfiers will work cross-platforms and cross-layout.

Control modifier works with almost all KeySym, on all layout and all OSes, but there are a few exceptions.

Future work

There are two main future work.

  1. Deprecation for char GetKeyCode() in favor of unsigned char GetKeyCode()
  2. Addition of unsigned char GetKeyCodeWithoutModifiers().
  3. Improve testing framework to test platform-specific implementation.

At this point, GetKeySym is enough for my needs, but if the need arise, I hope someone will implement these!

Thanks for the work that you have put into this, Mathieu. For future work, I would prefer to move away from using either char or unsigned char, since 256 codes is just not enough. Instead, I would like to get the actual text (as utf-8) generated by the key press:

  1. std::string GetKeyText()
  2. std::string GetKeyTextWithoutModifiers()

Unlike GetKeyCode(), this can easily be extended to all international keyboards and text input methods, without the need of large conversion tables inside VTK.

Hence, people could use GetKeySym() to check for special keys like ‘Up’, ‘Down’, ‘Delete’, but they would use GetKeyText() for ordinary keystrokes like ‘A’, ‘a’, ‘?’, or any unicode characters generated by international keyboards.

1 Like