Cannot get the correct AVI frame size after capturing multiple widget contents and outputing the AVI files

[cmodel]

The scenario is as the following:
I have separate qt widgets(different view for one object), which uses vtkWindow and vtkRenderer and other classes to display the correct 3D objects. Now I can run the animation action and we can see the object motions inside the widget. Now I want to record these motions from the whole different widgets. (e.g. I can see the motions from different view angle) into one single video file(AVI format).

In the first version, If I only output the animation from one qt widget. like I did as the following:

void startAnimationSave()
{
    vtkSmartPointer<vtkRenderWindow> renderWindow = leftWidget->getRenderer()->GetRenderWindow();
    saveFrame(renderWindow, filePath, mAviWriter);
}


void saveFrame(vtkSmartPointer<vtkRenderWindow> renderWindow, const QString& filePath, vtkAVIWriter* pAviWriter)
{
  renderWindow->SetSize(animationWidth, animationHeight);
  renderWindow->SetTileScale(scaleX, scaleY);
  renderWindow->OffscreenRenderingOn();
  renderWindow->Render();
  
  vtkNew<vtkWindowToImageFilter> imageFilter = vtkWindowToImageFilter::New();
  imageFilter->SetInput(renderWindow);
  
  if(nullptr == mAviWriter)
  {
     mAviWriter= vtkAVIWriter::New();
     mAviWriter->SetFileName(filePath);
     mAviWriter->SetInputConnection(imageFilter->GetOutputPort());
     mAviWriter->SetQuality(quality);
     mAviWriter->SetRate(frame);
     mAviWriter->Start();
  }
  mAviWriter->SetInputConnection(imageFilter->GetOutputPort());
  
  imageFilter->Modified();
  imageFilter->Update();
  mAviWriter->Write();
}

void stopAnimationSave()
{
    if(mAviWriter != nullptr)
    {
        mAviWriter->End();
    }
}

In each frame, we will call saveFrame() method, then it will wirte each frame image into AVI.

But if I want to render multiple widget motion pictures into one AVI video, I failed to get the correct effect. For the purpose of this, I wrote as the following:

1. Capture each image from different widgets.
2. Merge all these images into one single image with the correct layout.
3. All the images are saved into one std::vector<vtkImageData*>, for later writing out.
4. In the stopAnimationSave() method, I use the vtkImageCanvasSource2D and vtkImageViewer2 class to write each frame into AVI file.

Modify the stopAnimationSave() as the following:

void stopAnimationSave()
{
int* dims = mFrameImages[0]->GetDimensions();
int width = dims[0];
int height = dims[1];

vtkSmartPointer<vtkImageCanvasSource2D> source = vtkSmartPointer<vtkImageCanvasSource2D>::New();
source->SetScalarTypeToUnsignedChar();
source->SetNumberOfScalarComponents(3);
source->SetExtent(0, width-1, 0, height-1, 0, 0);

vtkSmartPointer<vtkAVIWriter> writer = vtkSmartPointer<vtkAVIWriter>::New();
vtkSmartPointer<vtkImageViewer2> imageViewer = vtkSmartPointer<vtkImageViewer2>::New();
imageViewer->SetInputConnection(source->GetOutputPort());
imageViewer->GetRenderer()->ResetCamera();
imageViewer->GetRenderer()->SetBackground(0, 0, 0);
imageViewer->SetSize(dims[0], dims[1]);

vtkSmartPointer<vtkWindowToImageFilter> fiter = vtkSmartPointer<vtkWindowToImageFilter>::New();
filter->SetInput(imageViewer->GetRenderWindow());
writer->SetInputConnection(filter->GetOutputPort());
imageViewer->GetRenderWindow()->OffScreenRenderingOn();

writer->SetFileName("d:/test.avi");
writer->SetRate(1);
writer->Start();

for(int i = 0; i < mFrameImages.size(); ++i)
{
 source->SetDrawColor(0, 0, 0);
 source->DrawImage(0, 0, mFrameImages[i]);
 source->Update();
 imageViewer->Render();
 filter->Modified();
 writer->Write();
}
writer->End();
}

The above code can generate one video file named test.avi and it has all frame images inside. But the canvas occupies the whole size of the video, and the frame image only fill in the center of the video, and there are much room around the centered frame image.

I want to each frame image fill the whole scene in the video, I have tried to change the size of the canvas, the size of the image viewer and the render window, but did not get the right result.

How can I do to fix this? Thanks.

[cmodel]

I have tried not using vtkImageViewer2, using the vtkRenderWindow to do the image displaying, I got the same result.

I attach the AVI screenshot here:

I need the frame image fill the whole display area, without the surrounding area.
Any idea how to fix this?

[jaswantp (Jaswant Panchumarti (Kitware))]

@cmodel

 source->DrawImage(0, 0, mFrameImages[i]);

i am not very familiar with the image canvas source class.

It looks like vtkImageCanvasSource2D can be used to merge all your images into one big image only if you pass the correct X and Y offsets for the first two args. You pass 0,0 in your code which just replaces the previous image.

[cmodel]

Frame images are correct for the generation of the AVI file, one image per frame, which I can see the animation effect from the output .avi file. I don’t need to merge all images together in each frame. Only need show one image for each frame.

I did not put the merged captured image for the case of multiple widgets. The merged captured image works fine in the code.
like the following:
The frame image is merged from two vertical widgets(capturing each image then merge in the right position).

The issue is the image in each frame did not fill the whole size.
Here we can see a large area of black region. I don’t need this black area. I have tried changing the size of canvas, the size of image, did not work. Who knows how to fix? Thanks.