How to force widget manager to use a particular renderer?

Are you trying to move the camera or the object? Based on your description, you want to modify the camera but your code move the object.

Then I don’t understand the intent behind those lines of code

const operationRenderer = orientationWidget.getRenderer();
orientationWidget.setParentRenderer();

axesActor.onModified(...);

Did you look at that widget example?