I struggled with this myself and ended up designing my own filter, which outputs data as a vtkRectilinearGrid for better performance (less memory). After seen this post, I made a small solution using vtkDepthImageToPointCloud. What made it work for me was to call Render before hooking up the rest of the pipeline. I have attached a small python snippet that works.
winColors = vtkRenderWindow()
winColors.SetSize(widthResolution, heightResolution)
winColors.SetOffScreenRendering(1)
renColors = vtkRenderer()
renColors.AddActor(yourActor)
winColors.AddRenderer(renColors)
winDepth = vtkRenderWindow()
winDepth.SetSize(widthResolution, heightResolution)
winDepth.SetOffScreenRendering(1)
renDepth = vtkRenderer()
renDepth.AddActor(yourActor)
winDepth.AddRenderer(renDepth)
winDepth.Render()
winColors.Render()
w2iColors = vtkWindowToImageFilter()
w2iColors.SetInput(winColors)
w2iColors.SetInputBufferTypeToRGBA()
w2iColors.Update()
w2iDepth = vtkWindowToImageFilter()
w2iDepth.SetInput(winDepth)
w2iDepth.SetInputBufferTypeToZBuffer()
w2iDepth.Update()
d2p = vtkDepthImageToPointCloud()
d2p.SetOutputPointsPrecision(0)
d2p.SetInputConnection(0, w2iDepth.GetOutputPort())
d2p.SetInputConnection(1, w2iColors.GetOutputPort())
d2p.SetCamera(yourCamera)
d2p.Update()
Note that the vtkDepthImageToPointCloud does not have points centered at the center of the pixels. There is a small offset. Unfortunately, you need to recompile the filter to fix this. I am considering making an update to this filter to also allow for a small vtkRectilinearGrid output, where colors as well as depths are stored in the point data.
It is enough with one shared renderer…