vtkBooleanOprationPolydataFilter does not work reliably. You may give a try to this other VTK Boolean operation filter: Adding Roemer's Boolean Filter as a Remote Module
vtkBooleanOprationPolydataFilter does not work reliably. You may give a try to this other VTK Boolean operation filter: Adding Roemer's Boolean Filter as a Remote Module