Hi, good morning, I’m trying to make a Depth-Map for Create Depth-Map for vtkDepthImageToPointCloud, for 3d reconstruction of jpeg or png images, but I can’t get the Depth-Map image to work as it should, I don’t know if I’m doing it right And I’m very frustrated because I can’t find how to do it, the image that I’m trying to create looks white and of course the 3d model is distorted, I would really appreciate some help, please.
std::string filename = "img/" + std::to_string(i) + ".jpg";
reader->SetFileName(filename.c_str());
reader->Update();
cast->SetInputData(reader->GetOutput());
cast->SetSplitModeToSlab();
cast->SetOutputScalarTypeToFloat();
cast->Update();
gaussianSmooth->SetInputData(cast->GetOutput());
gaussianSmooth->SetDimensionality(1);
gaussianSmooth->SetStandardDeviation(2.0);
gaussianSmooth->SetSplitModeToSlab();
const double derivations[3]{0.0, 30.0, 100.0};
gaussianSmooth->SetStandardDeviations(derivations);
gaussianSmooth->Update();
shiftScale->SetInputData(gaussianSmooth->GetOutput());
shiftScale->SetOutputScalarTypeToChar();
shiftScale->SetShift(0.0);
shiftScale->SetScale(255.0);
shiftScale->SetSplitModeToSlab();
shiftScale->Update();
depthImageToPointCloud->SetInputData(shiftScale->GetOutput());
depthImageToPointCloud->SetCamera(camera);
depthImageToPointCloud->CullNearPointsOn();
depthImageToPointCloud->CullFarPointsOn();
depthImageToPointCloud->Update();
Hello,
If I understood it right, you’re trying to get a depth map of some 3D scene and you’re getting a white image. So, before trying to save the rendered scene as image, what are you getting on screen?
best,
PC
Hello, what I want to do is a Depth-Map image, but I don’t have the results I want and I can’t find how to do it, then I’ll make the 3d model.
Hi,
How are you getting the depth buffer image?
Since the code posted seems to be incomplete, you can use the vtkWindowToImageFilter class (https://vtk.org/doc/nightly/html/classvtkWindowToImageFilter.html) to save z-buffer values as an image. Specifically, you need to call its SetInputBufferTypeToZBuffer() to obtain an image like this:
In the case above, the brighter the closer a pixel is (you may use another convention).
Also, please, check whether the solution to this question: https://discourse.vtk.org/t/z-buffer-values-problem-with-camera/2224/2 works for you. Likewise, the user was getting a blank image.
regards,
PC
I struggled with this myself and ended up designing my own filter, which outputs data as a vtkRectilinearGrid for better performance (less memory). After seen this post, I made a small solution using vtkDepthImageToPointCloud. What made it work for me was to call Render before hooking up the rest of the pipeline. I have attached a small python snippet that works.
winColors = vtkRenderWindow()
winColors.SetSize(widthResolution, heightResolution)
winColors.SetOffScreenRendering(1)
renColors = vtkRenderer()
renColors.AddActor(yourActor)
winColors.AddRenderer(renColors)
winDepth = vtkRenderWindow()
winDepth.SetSize(widthResolution, heightResolution)
winDepth.SetOffScreenRendering(1)
renDepth = vtkRenderer()
renDepth.AddActor(yourActor)
winDepth.AddRenderer(renDepth)
winDepth.Render()
winColors.Render()
w2iColors = vtkWindowToImageFilter()
w2iColors.SetInput(winColors)
w2iColors.SetInputBufferTypeToRGBA()
w2iColors.Update()
w2iDepth = vtkWindowToImageFilter()
w2iDepth.SetInput(winDepth)
w2iDepth.SetInputBufferTypeToZBuffer()
w2iDepth.Update()
d2p = vtkDepthImageToPointCloud()
d2p.SetOutputPointsPrecision(0)
d2p.SetInputConnection(0, w2iDepth.GetOutputPort())
d2p.SetInputConnection(1, w2iColors.GetOutputPort())
d2p.SetCamera(yourCamera)
d2p.Update()
Note that the vtkDepthImageToPointCloud does not have points centered at the center of the pixels. There is a small offset. Unfortunately, you need to recompile the filter to fix this. I am considering making an update to this filter to also allow for a small vtkRectilinearGrid output, where colors as well as depths are stored in the point data.
It is enough with one shared renderer…
Hello, thanks for the help to both of you, I’ll try to see if I get it, thanks.
I realized now why I created my own filter. I never succeeded in getting more than one output from filter. I have tried manually updating the previous filters and explicitly calling Render on the render windows.
Hello, here is everything I try to do to create the depth map, but I get an error that I don’t know why, the image is not because I tried with several images, and I have it right in the directory.
#include <iostream>
#include "vtkActor.h"
#include <vtkNamedColors.h>
#include <vtkRenderWindow.h>
#include <vtkRenderWindowInteractor.h>
#include <vtkRenderer.h>
#include <vtkSmartPointer.h>
#include "vtkLight.h"
#include "vtkCamera.h"
#include <vtkDepthImageToPointCloud.h>
#include <vtkInteractorStyleTrackballCamera.h>
#include <vtkWindowToImageFilter.h>
#include <vtkAutoInit.h>
#include "vtkTexture.h"
#include <vtkImageViewer2.h>
#include <vtkPNGReader.h>
VTK_MODULE_INIT(vtkRenderingOpenGL2);
VTK_MODULE_INIT(vtkInteractionStyle);
int main(int argc, char* argv[])
{
vtkSmartPointer<vtkActor> actor = vtkSmartPointer<vtkActor>::New();
vtkSmartPointer<vtkPNGReader> reader = vtkSmartPointer<vtkPNGReader>::New();
vtkSmartPointer<vtkCamera> camera = vtkSmartPointer<vtkCamera>::New();
vtkSmartPointer<vtkLight> light = vtkSmartPointer<vtkLight>::New();
vtkSmartPointer<vtkNamedColors> colorsFondo = vtkSmartPointer<vtkNamedColors>::New();
vtkSmartPointer<vtkNamedColors> colors = vtkSmartPointer<vtkNamedColors>::New();
vtkSmartPointer<vtkRenderWindowInteractor> interactor = vtkSmartPointer<vtkRenderWindowInteractor>::New();
vtkSmartPointer<vtkInteractorStyleTrackballCamera> interactorStyle = vtkSmartPointer<vtkInteractorStyleTrackballCamera>::New();
vtkSmartPointer<vtkTexture> texture = vtkSmartPointer<vtkTexture>::New();
vtkSmartPointer<vtkRenderWindow> renderWindow = vtkSmartPointer<vtkRenderWindow>::New();
vtkSmartPointer<vtkRenderer> render = vtkSmartPointer<vtkRenderer>::New();
vtkSmartPointer<vtkWindowToImageFilter> windowToImageFilter = vtkSmartPointer<vtkWindowToImageFilter>::New();
//vtkSmartPointer<vtkDepthImageToPointCloud> depthImageToPointCloud = vtkSmartPointer<vtkDepthImageToPointCloud>::New();
try
{
reader->SetFileName("foto.png");
reader->Update();
texture->SetInputConnection(reader->GetOutputPort());
texture->SetQualityToDefault();
texture->Update();
camera->SetPosition(0, 0, 500);
camera->SetFocalPoint(0, 0, 0);
light->SetPosition(0, 0, 500);
light->SetDiffuseColor(colors->GetColor3d("White").GetData());
light->SetSpecularColor(colors->GetColor3d("White").GetData());
light->SetIntensity(1.0);
actor->SetTexture(texture);
actor->SetCoordinateSystemToWorld();
windowToImageFilter->SetInput(renderWindow);
windowToImageFilter->SetInputBufferTypeToZBuffer();
windowToImageFilter->SetReadFrontBuffer(false);
windowToImageFilter->SetProgressShiftScale(-1.0, 0.02);
windowToImageFilter->Update();
/*
depthImageToPointCloud->SetOutputPointsPrecision(0);
depthImageToPointCloud->SetInputConnection(0, windowToImageFilter->GetOutputPort());
depthImageToPointCloud->SetCamera(camera);
depthImageToPointCloud->Update();
*/
render->AddActor(actor);
render->AddLight(light);
render->GetActiveCamera();
renderWindow->AddRenderer(render);
renderWindow->Render();
renderWindow->SetSize(800, 800);
renderWindow->SetWindowName("depth-map image");
interactor->SetInteractorStyle(interactorStyle);
interactor->SetRenderWindow(renderWindow);
interactor->Initialize();
interactor->Start();
}
catch (const std::exception& e) {
std::cerr << "Error: " << e.what() << std::endl;
return EXIT_FAILURE;
}
catch (...) {
std::cerr << "Error desconocido." << std::endl;
return EXIT_FAILURE;
}
return 0;
}
ERROR: In vtkImageData.cxx, line 1548
vtkImageData (000002708AA701A0): GetScalarPointer: Pixel (0, 0, 0) not in memory.
Current extent= (0, -1, 0, -1, 0, 0)
It is not strange that this doesn’t work. When you update the vtkWindowToImageFilter the renderWindow has no renderer. Try to translate what wrote above and you will get a point cloud.