How to build VTK to use x11 and opengles as rendering backend in an embedded system

Support
1

I’m trying to build VTK to use x11 and opengles as rendering backend in an embedded system(RK3588). I can’t use the EGL pipeline because my board doesn’t support EGL, but it supports x11. So I tried to build VTK with VTK_OPENGL_USE_GLES = ON and VTK_USE_X = ON , it builds with no error. However when I try to use it for volumerending, it works fine but I don’t think GPU is used. The Example I use is nearly the same as SimpleRayCast. I add some code to debug, I find vtkXOpenGLRenderWindow is already being used.

  std::cout << "GetNumberOfDevices : " << renWin->GetNumberOfDevices() << std::endl;
  std::cout << "GetRenderingBackend : " << renWin->GetRenderingBackend() << std::endl;
  vtkXOpenGLRenderWindow *glRenWin =
      vtkXOpenGLRenderWindow::SafeDownCast(renWin);

  std::cout << "renWin->GetClassName()  :  "<< renWin->GetClassName()<< std::endl;
  if(glRenWin)
  {
    int major;
    int minor;
    glRenWin->GetOpenGLVersion(major, minor);
    //std::cout << glRenWin->ReportCapabilities() << std::endl;
    std::cout << glRenWin->GetOpenGLSupportMessage() << std::endl;
    std::cout << glRenWin->SupportsOpenGL() << std::endl;
    std::cout << "OpenGL : "<<major << " . " << minor << std::endl;
  }

The code above print out something like:

GetNumberOfDevices : 0
GetRenderingBackend : OpenGL2
renWin->GetClassName()  :  vtkXOpenGLRenderWindow
Not tested yet
1
OpenGL : 4 . 5

I think the OpenGL version should be something like 3.2(opengles)
I run glmark2 on my system and it print out something like:

forlinx@ok3588:~$ glmark2
arm_release_ver of this libmali is 'g6p0-01eac0', rk_so_ver is '6'.
=======================================================
    glmark2 2021.12
=======================================================
    OpenGL Information
    GL_VENDOR:     ARM
    GL_RENDERER:   Mali-LODX
    GL_VERSION:    OpenGL ES 3.2 v1.g6p0-01eac0.ba52c908d926792b8f5fe28f383a2b03
=======================================================
[build] use-vbo=false:^C FPS: 1471 FrameTime: 0.680 ms
=======================================================
                                  glmark2 Score: 1471 
=======================================================

glxinfo is something like:

forlinx@ok3588:~$ glxinfo -B
name of display: :0
display: :0  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: Mesa/X.org (0xffffffff)
    Device: llvmpipe (LLVM 12.0.0, 128 bits) (0xffffffff)
    Version: 21.2.6
    Accelerated: no
    Video memory: 7610MB
    Unified memory: no
    Preferred profile: core (0x1)
    Max core profile version: 4.5
    Max compat profile version: 3.1
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.2
OpenGL vendor string: Mesa/X.org
OpenGL renderer string: llvmpipe (LLVM 12.0.0, 128 bits)
OpenGL core profile version string: 4.5 (Core Profile) Mesa 21.2.6
OpenGL core profile shading language version string: 4.50
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 3.1 Mesa 21.2.6
OpenGL shading language version string: 1.40
OpenGL context flags: (none)

OpenGL ES profile version string: OpenGL ES 3.2 Mesa 21.2.6
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20

So where am I doing wrong? Or It just doesn’t work like that.

2

@sankhesh

3

@ltylight Please switch to using the vtkSmartVolumeMapper in the example code and test again.

4

I think the GPU is not in use. I use the command below to watch if GPU is in use.

watch -n 0.1 cat /sys/devices/platform/fb000000.gpu/devfreq/fb000000.gpu/load

When I run gmark2, the gpu load is among 50 to 80, but when I run my vtk example, the gpu load is among 0 to 10.
The code below is what I use.

//#define OPENGL_ES_VERSION 3


#include <vtkCamera.h>
#include <vtkColorTransferFunction.h>
#include <vtkFixedPointVolumeRayCastMapper.h>
#include <vtkNamedColors.h>
#include <vtkNew.h>
#include <vtkPiecewiseFunction.h>
#include <vtkRenderWindow.h>
#include <vtkRenderWindowInteractor.h>
#include <vtkRenderer.h>
#include <vtkStructuredPointsReader.h>
#include <vtkInteractorStyleTrackballCamera.h>
#include <vtkVolume.h>
#include <vtkMetaImageReader.h>
#include <vtkVolumeProperty.h>
#include <vtkOpenGLRenderWindow.h>
#include <vtkNIFTIImageReader.h>
#include <vtkPolyDataMapper.h>
#include <vtkActor.h>
#include <vtkConeSource.h>
#include <vtkAutoInit.h>
#include <vtkOpenGLRenderer.h>
#include <vtkXOpenGLRenderWindow.h>
#include <vtkSmartVolumeMapper.h>

#include <X11/Xlib.h> // Needed for X types used in the public interface
#include <X11/Xutil.h> // Needed for X types used in the public interface

VTK_MODULE_INIT(vtkRenderingOpenGL2);
//VTK_MODULE_INIT(vtkRenderingFreeType);
VTK_MODULE_INIT(vtkInteractionStyle);
VTK_MODULE_INIT(vtkRenderingVolumeOpenGL2);

int main(int argc, char *argv[])
{
  vtkNew<vtkNamedColors> colors;

  vtkNew<vtkRenderer> ren1;

  vtkNew<vtkRenderWindow> renWin;
  renWin->AddRenderer(ren1);

  vtkNew<vtkRenderWindowInteractor> iren;
  iren->SetRenderWindow(renWin);
  iren->SetDesiredUpdateRate(0);

  vtkNew<vtkInteractorStyleTrackballCamera> istyle;
  iren->SetInteractorStyle(istyle);

  // Create the reader for the data
  // vtkNew<vtkStructuredPointsReader> reader;
  // reader->SetFileName("/userdata/DicomData/ironProt.vtk");

  // vtkSmartPointer<vtkImageData> input;
  // vtkNew<vtkMetaImageReader> reader;
  // reader->SetFileName("/userdata/DicomData/test.mha");
  // reader->Update();

  vtkSmartPointer<vtkNIFTIImageReader> reader =
      vtkSmartPointer<vtkNIFTIImageReader>::New();
  reader->SetFileName("/userdata/DicomData/NII/resampleAndPoints_ori.nii");
  reader->Update();

  vtkNew<vtkColorTransferFunction> colorFun;
  vtkNew<vtkPiecewiseFunction> opacityFun;
  colorFun->AddRGBPoint(-3024, 0, 0, 0, 0.5, 0.0);
  colorFun->AddRGBPoint(-16, 0.73, 0.25, 0.30, 0.49, .61);
  colorFun->AddRGBPoint(641, .90, .82, .56, .5, 0.0);
  colorFun->AddRGBPoint(3071, 1, 1, 1, .5, 0.0);

  opacityFun->AddPoint(-3024, 0, 0.5, 0.0);
  opacityFun->AddPoint(-16, 0, .49, .61);
  opacityFun->AddPoint(641, .72, .5, 0.0);
  opacityFun->AddPoint(3071, .71, 0.5, 0.0);

  // The property describes how the data will look
  vtkNew<vtkVolumeProperty> volumeProperty;
  volumeProperty->SetColor(colorFun);
  volumeProperty->SetScalarOpacity(opacityFun);
  volumeProperty->ShadeOn();
  volumeProperty->SetInterpolationTypeToLinear();

  // The mapper / ray cast function know how to render the data
  //vtkNew<vtkFixedPointVolumeRayCastMapper> volumeMapper;
  vtkNew<vtkSmartVolumeMapper> volumeMapper;
  
  volumeMapper->SetInputConnection(reader->GetOutputPort());

  // The volume holds the mapper and the property and
  // can be used to position/orient the volume
  vtkNew<vtkVolume> volume;
  volume->SetMapper(volumeMapper);
  volume->SetProperty(volumeProperty);

  ren1->AddVolume(volume);

  vtkNew<vtkConeSource> cone;
  cone->SetHeight(3.0);
  cone->SetRadius(1.0);
  cone->SetResolution(10);
  vtkNew<vtkPolyDataMapper> coneMapper;
  coneMapper->SetInputConnection(cone->GetOutputPort());
  vtkNew<vtkActor> coneActor;
  coneActor->SetMapper(coneMapper);

  ren1->AddActor(coneActor);

  ren1->SetBackground(colors->GetColor3d("Wheat").GetData());
  ren1->GetActiveCamera()->Azimuth(45);
  ren1->GetActiveCamera()->Elevation(30);
  ren1->ResetCameraClippingRange();
  ren1->ResetCamera();

  renWin->SetSize(600, 600);
  renWin->SetWindowName("SimpleRayCast");
  renWin->Render();

  std::cout << "GetLastUsedRenderMode : "<<volumeMapper->GetLastUsedRenderMode()<<" " << vtkSmartVolumeMapper::GPURenderMode << std::endl;
  std::cout << "GetNumberOfDevices : " << renWin->GetNumberOfDevices() << std::endl;
  std::cout << "GetRenderingBackend : " << renWin->GetRenderingBackend() << std::endl;
  vtkXOpenGLRenderWindow *glRenWin =
      vtkXOpenGLRenderWindow::SafeDownCast(renWin);

  std::cout << "renWin->GetClassName()  :  "<< renWin->GetClassName()<< std::endl;
  if(glRenWin)
  {
    int major;
    int minor;
    glRenWin->GetOpenGLVersion(major, minor);
    //std::cout << glRenWin->ReportCapabilities() << std::endl;
    std::cout << glRenWin->GetOpenGLSupportMessage() << std::endl;
    std::cout << glRenWin->SupportsOpenGL() << std::endl;
    std::cout << "OpenGL : "<<major << " . " << minor << std::endl;
  }
  

  iren->Start();

  return EXIT_SUCCESS;
}

And the output of the code is

GetLastUsedRenderMode : 2 2
GetNumberOfDevices : 0
GetRenderingBackend : OpenGL2
renWin->GetClassName()  :  vtkXOpenGLRenderWindow
Not tested yet
1
OpenGL : 4 . 5
5

When I run gmark2, the gpu load is among 50 to 80, but when I run my vtk example, the gpu load is among 0 to 10.

That should be fine. gmark2 is designed to test heavy load.

To be more sure, switch to using the vtkGPUVolumeRayCastMapper instead.