That’s the correct approach.
That’s actually your only option. COM was designed to permit passing objects (interfaces actually) across library boundaries. Doing it in C++ is nothing but a hack in my opinion.
Using text as a key in the colour map is another problem. It should be an enumeration or integer key mapped to names.