vtkCamera SetParallelProjection not working properly

Dear VTK community,

I am trying to change vtk camera from parallel mode to perspective mode by using SetParallelProjection(True/False), everything works as expected until I scroll the mouse, which changes the view scale. The view scale is not reflected when change the camera projection mode (as depicted by the pic below). Does anybody know how to solve this problem?