Output video of vtkFFMPEGWriter/any video writer plays too fast

Hi All,

Environment:
VTK 8.2
.NET 4.5.2
Windows 10

Using VTK 8.2, the output video from all video writers (vtkAVIWriter, vtkOggTheoraWriter, and my preference vtkFFMPEGWriter) plays too fast. It seems that frame rate is not set correctly since it is always 600 fps. referring to the similar topic in Development section

Cannot modify frame rate for a video using vtkFFMPEGWriter class

seems that pts is not shipped correctly with frame data and therefore timestamps of frames are incorrect to my understanding.
I nailed to temporarily fix the problem, when I changed the renderer, renderWindow, and renderWindowInteractor from OpenGL types to generic defaults ones. But, after few time of building my project in different profiles, the problem returned and persists. Here is the code snippets I am using to render and record. Render works on timer basis, on each render event, video writer and filter update and write.

int Visulization setup()
    {
		.
		.
		.
		this->isRecording == false
		// Create a renderer, render window, and interactor
		vtkData->renderer = vtkSmartPointer<vtkRenderer>::New();
		vtkData->renderWindow = vtkRenderWindow::New();
		vtkData->renderWindowInteractor = vtkRenderWindowInteractor::New();
       
		// Create Writer
		vtkData->w2iFilter = vtkWindowToImageFilter::New();
		vtkData->w2iFilter->SetInput(vtkData->renderWindow);

		vtkData->videoWriter= vtkSmartPointer<vtkFFMPEGWriter>::New();
		vtkData->videoWriter->SetFileName("d://animation.avi");
		/*vtkData->videoWriter->SetQuality(2);
		vtkData->videoWriter->SetRate(15);*/
		vtkData->videoWriter->SetInputConnection(vtkData->w2iFilter->GetOutputPort());
		.
		.
		.
		
	}

void RenderWithFrameTimer::RenderCallback()
	{
		while (!disabled && !closed)
		{
			this->dispatcher->Invoke(performRenderDelegate);
			this->onRenderEvent();
			Tasks::Task::Delay(25)->Wait();
		}
	}
	
void RenderWithFrameTimer::performRenderDelegate()
	{
		renderMutex->WaitOne();
		if (!disabled && !closed && renderThread->IsAlive)
		{
			renderWindow->Render();
		}
		renderMutex->ReleaseMutex();
	}
				
void VtkSimulationWindow::startRecording()
	{
		if (this->isRecording == false)
		{
			vtkData->videoWriter->Start();
			isRecording = true;
		}
	}
	
void VtkSimulationWindow::stopRecording()
	{
		if (this->isRecording == true)
		{
			vtkData->videoWriter->End();
			isRecording = false;
		}		
	}
	
void VtkSimulationWindow::onRenderEvent()
	{
		if (isRecording) {
			captureFrame();
		}
	}
	
void VtkSimulationWindow::captureFrame()
	{
		if (this->isRecording == true)
		{
			vtkData->w2iFilter->Modified();
			vtkData->w2iFilter->Update();
			vtkData->videoWriter->Write();
		}
	}

I have also tried to replace the source of data to get directly from renderer using vtkRendererSource, and vtkRenderLargeImage, however, it did not made any difference.

@ken-martin

1 Like

I have just found the solution.
The issue lies where time_base of avStream is not set in the vtkFFMPEGWriterInternal class, when Start() method is called. Therefore, due to using libavformat, and not having VFR flag set to dynamically adapt framerate and time_base (not sure that would be an alternative!, but for sure involved in this issue), the constant frame time equals to default avStream->time_base (which is = 1/FPS default = 1/600) is used to pass the input stream to encoder. Adding following two lines of code:

this->avStream->time_base.den = this->FrameRate;
this->avStream->time_base.num = 1;

after instantiating stream in following line in Start() method:

Preformatted textthis->avStream = avfPreformatted textormat_new_stream(this->avFormatContext, 0);

also set the framerate for codec, and adapt to new API in case of using recent ffmpeg library:

#if LIBAVFORMAT_VERSION_INT < AV_VERSION_INT(57,37,100)
  int got_frame;
  int ret = avcodec_encode_video2(cc,
                                  &pkt,
                                  this->yuvOutput,
                                  &got_frame);

  //dump the compressed result to file
//dump the compressed result to file
  if (got_frame)
  {
    pkt.stream_index = this->avStream->index;
    ret = av_write_frame(this->avFormatContext, &pkt);
	  pkt.stream_index = this->avStream->index;
	  ret = av_write_frame(this->avFormatContext, &pkt);
  }
#else
  cc->framerate.num = this->FrameRate;
  cc->framerate.den = 1;

  int ret = avcodec_send_frame(cc,this->yuvOutput);
  if (ret<0)
  {
	  vtkGenericWarningMacro(<< "Problem encoding frame.");
	  return 0;
  }
  ret = avcodec_receive_packet(cc, &pkt);
#endif

will solve the problem.
Refer to merge request.

Do you mind sending a merge request with the diff so that this doesn’t affect anyone else in the future? @ken-martin

I have just made the merge request, but it seems that it cannot be targeted to version 8.2.0.

This adaptation seems to be done in current version 9.0d, but I made a very quick test, and it got a wrong FPS 600 using ffmpeg-4.2.2, must probably because time_base is still not set for codec.

Ah, I missed that this was VTK 8.2. There won’t be anymore 8.2.x releases, so could you please rebase onto master (or release to get it into 9.0)?