VTK with PCL not working as expected with migration to qt6 QVTKWidget to QVTKOpenGLNativeWidget

Hello guys ,

I am migrating a desktop based application from qt 5 to qt 6 which includes following libraries

PCL
VTK
Qt

Due to deprication and compatible issues I was forced to upgrade VTK and PCL to VTK 9.1 and PCL 1.15 . I was using QVTKWidget to visualize some LIDAR data (.pcd files) now during migration I learnt QVTKWidget has a new advanced widget called QVTKOpenGLNativeWidget . I have integrated with the new widget and I am facing following issues .

  1. Two renderers are opening up . One from the PCL and one from QVTKOpenGLNativeWidget .

  2. PCL renderer is being poped up as a seperate window and LIDAR data is visualized there and QVTKOpenGLNativeWidget intigrated to the application is blank .

mainwindow.cpp

MainWindow::MainWindow(QWidget *parent)
    : QMainWindow(parent)
    , ui(new Ui::MainWindow)
{
    QProcessEnvironment env = QProcessEnvironment::systemEnvironment();
    QString username = env.value("USER");

    ui->setupUi(this);
    loadCamParameters();

    currentDirPath=QDir::currentPath();
    ui->stackedWidget->setCurrentIndex(2);
    this->setWindowFlag(Qt::WindowMinMaxButtonsHint, false);
    SetDisableEnable(true);
    ui->bplay->setDisabled(true);
    ResettingCloudPoints();
    WidgetPictureUpdate();
}

void MainWindow::ResettingCloudPoints()
{
    input_cloud.reset(new PointCloudT);

    // 1. Create viewer
    viewer_3D.reset(new pcl::visualization::PCLVisualizer("viewer_3D", true));

    // 2. Create VTK-compatible render window
    vtkSmartPointer<vtkGenericOpenGLRenderWindow> renderWindow =
        vtkSmartPointer<vtkGenericOpenGLRenderWindow>::New();

    // 3. Assign to QVTK widget
    ui->PCD_Viewer->setRenderWindow(renderWindow);

    // 4. Connect PCL viewer to Qt's interactor
    viewer_3D->setupInteractor(ui->PCD_Viewer->interactor(), ui->PCD_Viewer->renderWindow());

    // 5. Optional: position camera
    viewer_3D->initCameraParameters();

    ui->PCD_Viewer->update();

    setCameraParams();
}

.ui file

    <widget class="QWidget" name="DVTPage">
     <property name="styleSheet">
      <string notr="true">QPushButton::hover{background-color:rgb(250, 233, 79);color:rgb(250, 233, 79);}
border: 2px solid black;</string>
     </property>
     <widget class="QVTKOpenGLNativeWidget" name="PCD_Viewer" native="true">
      <property name="enabled">
       <bool>true</bool>
      </property>
      <property name="geometry">
       <rect>
        <x>10</x>
        <y>80</y>
        <width>361</width>
        <height>321</height>
       </rect>
      </property>
      <property name="styleSheet">
       <string notr="true">
border: 2px solid rgb(79, 192, 232);</string>
      </property>
      <widget class="QLabel" name="label_5">
       <property name="geometry">
        <rect>
         <x>0</x>
         <y>0</y>
         <width>61</width>
         <height>21</height>
        </rect>
       </property>
       <property name="styleSheet">
        <string notr="true">background-color: rgb(211, 215, 207);
border: 2px solid black;</string>
       </property>
       <property name="text">
        <string>LIDAR</string>
       </property>
      </widget>
      <widget class="QLabel" name="FrameInfo">
       <property name="geometry">
        <rect>
         <x>290</x>
         <y>300</y>
         <width>71</width>
         <height>20</height>
        </rect>
       </property>
       <property name="styleSheet">
        <string notr="true">background-color: rgb(255, 255, 255);
border: 2px solid black;</string>
       </property>
       <property name="text">
        <string/>
       </property>
      </widget>
     </widget>


<customwidget>
   <class>QVTKOpenGLNativeWidget</class>
   <extends>QWidget</extends>
   <header location="global">QVTKOpenGLNativeWidget.h</header>
   <container>1</container>
  </customwidget>
 </customwidgets>

I have multiple questions (some of them might find silly ) as I am very new VTK and PCL .Here it goes :

  1. As far I have know PCL cannot render itself , it needs VTK or other 3rd party libs to render into something . But how in this case a new window is poping up visualized there where as the widget which I promoted to QVTKOpenGLNativeWidget is blank . I have checked with logging
qDebug() << "viewer getRenderWindow() =" << viewer_3D->getRenderWindow();
qDebug() << "widget renderWindow() =" << ui->PCD_Viewer->renderWindow();

I have got something like viewer getRenderWindow() = 0x5601897e0810 widget renderWindow() = 0x5601897e0810’

  1. Previously PCL Visualizer can directly be integrated with QVTKWidget , but now QVTKOpenGLNativeWidget has its own renderer and I am not sure how integrate PCL visualizer to the new widget.

I have also tried with a sample example

mainwindow.cpp

#include "mainwindow.h"
#include "./ui_mainwindow.h"

MainWindow::MainWindow(QWidget *parent)
    : QMainWindow(parent)
    , ui(new Ui::MainWindow)
{
    ui->setupUi(this);
    vtkSmartPointer<vtkRenderWindow> embeddedRenderWindow = ui->PCD_Viewer->renderWindow();
    vtkSmartPointer<vtkRenderWindowInteractor> interactor = ui->PCD_Viewer->interactor();
    viewer_3D.reset(new pcl::visualization::PCLVisualizer("PCD Viewer", false));

    vtkSmartPointer<vtkRenderer> pclRenderer = viewer_3D->getRendererCollection()->GetFirstRenderer();

    embeddedRenderWindow->AddRenderer(pclRenderer);

    if (!embeddedRenderWindow || !interactor) {
        qDebug() << "ERROR: VTK Render Window or Interactor is null. Check UI setup and VTK versions.";
        // Stop execution or handle error gracefully
        //return;
    }

    // 6. Set up the Interactor (Telling PCL to use the widget's interactor and the embedded window)
    viewer_3D->setupInteractor(interactor, embeddedRenderWindow);

    // 7. Final setup and rendering
    viewer_3D->addCoordinateSystem(1.0);
    viewer_3D->setBackgroundColor(0, 0, 0);
    viewer_3D->resetCamera();

    interactor->Initialize();
    embeddedRenderWindow->Render();

    current_cloud.reset(new PointCloudT);

    // Add an empty cloud initially to the viewer with an ID
    viewer_3D->addPointCloud<PointT>(current_cloud, "pcd_cloud_id");
    viewer_3D->resetCamera();

    on_actionLoad_PCD_triggered();

    // pclUpdateTimer = new QTimer(this);
    // connect(pclUpdateTimer, &QTimer::timeout, this, &MainWindow::updatePCLViewer);
    // pclUpdateTimer->start(10);

    // Connect UI elements (assuming an action named actionLoad_PCD in your UI)
    //connect(ui->actionLoad_PCD, &QAction::triggered, this, &MainWindow::on_actionLoad_PCD_triggered);

    pclUpdateTimer = new QTimer(this);
    connect(pclUpdateTimer, &QTimer::timeout, this, &MainWindow::updatePCLViewer);
    pclUpdateTimer->start(10);
}

void MainWindow::fileSelect()
{
    dir= QFileDialog::getExistingDirectory(this,"Choose Input PCAP / PCD Folder","currentDirPath",  QFileDialog::DontResolveSymlinks | QFileDialog::ReadOnly);
    if(dir.isEmpty())
    {
        qDebug()<<"No folder is selected";
        return;
    }
    //Fetching Directory Path
    QDir directory=QDir(dir);
    inputFiles = directory.entryList(QStringList() <<"*.pcap"<<"*.pcd"<<"*.csv",QDir::Files);

    //Calculating total number of Lidar Frame
    totalNumberofLidarFrames=inputFiles.length();

    // qDebug("%d", totalNumberofLidarFrames);
    if(totalNumberofLidarFrames==0)
    {
        qDebug()<<"No Input PCAP/ PCD Files are There";
        //ui->bFile->setHidden(false);
        return;
    }

    /*Images and Annotation path Filtering*/
    anotation_Path=dir.section("/",0,-2)+"/"+"txt_results"; //ELSE CASE
    imagePath=dir.section("/",0,-2)+"/"+"original_images"; //ELSE CASE

    if(!((QDir(anotation_Path).exists())&&(QDir(imagePath).exists())))
    {
        qDebug()<<"Either original_images or txt_results folder is missing ";
        //ui->bFile->setHidden(false);
        return;
    }
    //Fetching images from directory
    QDir imgdirectory(imagePath);
    imageFiles = imgdirectory.entryList(QStringList() <<"jpeg"<<"*.jpg"<<"*.png"<<"*.bmp",QDir::Files);

    if(imageFiles.length())
    {
        //totalNumberofFrames=imageFiles.length(); //Image Else Case Had to add
    }
    else
    {
        //AckMessage("No Input Images are There");
        //ui->bFile->setHidden(false);
        return;
    }

    /*Widgets enable and disable functionality*/
    //SetDisableEnable(false);
    //ui->bplay->setDisabled(false);
    //Synchronization dialog box for DN8 Camera
    //if(!isGL3)
    // {
    //     DN8_Syn_Frame();
    // }

    //Frame processing functionality
    //FrameProcessing();
}

void MainWindow::loadAndVisualizePCD(const std::string &filename)
{
    // Clear existing data before loading new data (optional but good practice)
    current_cloud->clear();

    // Use PCL's IO function to load the file
    if (pcl::io::loadPCDFile<PointT>(filename, *current_cloud) == -1)
    {
        PCL_ERROR("Error loading PCD file %s\n", filename.c_str());
        QMessageBox::critical(this, "Error", QString("Could not load file: %1").arg(QString::fromStdString(filename)));
        return;
    }

    // Successfully loaded, now update the visualizer
    if (current_cloud->size() > 0)
    {
        // Update the existing cloud in the viewer using its ID ("pcd_cloud_id")
        viewer_3D->updatePointCloud(current_cloud, "pcd_cloud_id");

        // Optional: set point size
        viewer_3D->setPointCloudRenderingProperties(
            pcl::visualization::PCL_VISUALIZER_POINT_SIZE,
            5,
            "pcd_cloud_id"
            );

        // Reset the camera view to fit the new cloud data
        viewer_3D->resetCamera();

        // Explicitly trigger a render update for the widget
        ui->PCD_Viewer->update();
        ui->PCD_Viewer->renderWindow()->Render();

        qDebug() << "Successfully loaded and visualized " << current_cloud->size() << " points.";
    }
}

MainWindow::~MainWindow()
{
    delete ui;
}

void MainWindow::updatePCLViewer()
{

}

void MainWindow::on_actionLoad_PCD_triggered()
{
    // Use QFileDialog to get a file path from the user
    QString q_filename = QFileDialog::getOpenFileName(
        this,
        tr("Open Point Cloud File"),
        "./", // Start in current directory
        tr("Point Cloud Data (*.pcd);;All Files (*.*)")
        );

    if (!q_filename.isEmpty()) {
        loadAndVisualizePCD(q_filename.toStdString());
    }
}


.ui

<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
 <class>MainWindow</class>
 <widget class="QMainWindow" name="MainWindow">
  <property name="geometry">
   <rect>
    <x>0</x>
    <y>0</y>
    <width>800</width>
    <height>600</height>
   </rect>
  </property>
  <property name="windowTitle">
   <string>MainWindow</string>
  </property>
  <widget class="QWidget" name="centralwidget">
   <widget class="QVTKOpenGLNativeWidget" name="PCD_Viewer" native="true">
    <property name="geometry">
     <rect>
      <x>200</x>
      <y>70</y>
      <width>500</width>
      <height>250</height>
     </rect>
    </property>
   </widget>
  </widget>
  <widget class="QMenuBar" name="menubar">
   <property name="geometry">
    <rect>
     <x>0</x>
     <y>0</y>
     <width>800</width>
     <height>22</height>
    </rect>
   </property>
  </widget>
  <widget class="QStatusBar" name="statusbar"/>
 </widget>
 <customwidgets>
  <customwidget>
   <class>QVTKOpenGLNativeWidget</class>
   <extends>QWidget</extends>
   <header location="global">QVTKOpenGLNativeWidget.h</header>
   <container>1</container>
  </customwidget>
 </customwidgets>
 <resources/>
 <connections/>
</ui>


which turned out to be not opening any window but didnt render / visualize the point cloud . Thanks for the help in advance ..