vtkInteractorStyleRubberBand3D not playing nice with QQuickVTKRenderWindow

I was playing around with QQuickVTKRenderWindow. Instead of a trackball interactor on line 260, I put in a vtkInteractorStyleRubberBand3D. But when I try using it, it throws an error

vtkGenericOpenGLRenderWindow (0000024B9236C8E0): ReadPixels called before window is ready for rendering;

This is due to vtkInteractorStyleRubberBand3D calling the method GetRGBACharPixelData which calls ReadPixels. If I keep on clicking I finally get an error:

vtkOpenGLState.cxx:68 WARN| Error in cache state for GL_DEPTH_WRITEMASK

I notice the ReadPixels Warning is triggered because the ReadyForRendering state is set to false after assigning the interactor at line 263. Why is this property been set to false? When I set them all to true, I just get the GL_DEPTH_WRITEMASK error.

I get the feeling their might some synchronization issue between the qt quick opengl calls and the vtk.

EDIT : I just found that it is sort of related to this issue.