We use ParaView in that project, but that is the same gist with VTK. In fact, you can even use that ParaView to run your VTK code.
Currently we don’t provide EGL wheels of VTK (but they are WIP).
Hi @Sebastien_Jourdain ,
Can you please share where is the Dockerfile to create image kitware/trame:glvnd ? I saw the image in the dockerhub at : Docker . However I want to know your Dockerfile which creates that image .
Thanks
The docker files are captured in trame and the CI build a matrix of images.
Hi Sebastien,
I went to https://github.com/Kitware/paraview-visualizer/tree/master/docker to see what you did setup for EGL and GPU. I got this error, can you please help to elaborate what do I need to set it up correctly ?
~/workingspace/paraview-visualizer/docker ~/workingspace/paraview-visualizer/docker
~/workingspace/paraview-visualizer/docker
* Restarting Apache httpd web server apache2
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
...done.
Starting server...
/opt/trame/activate_venv.sh: line 1: /deploy/server/venv/bin/activate: No such file or directory
/opt/trame/run.sh: line 20: /deploy/server/launcher.json: No such file or directory
Starting the wslink launcher at
/usr/bin/python: Error while finding module specification for 'wslink.launcher' (ModuleNotFoundError: No module named 'wslink')
[+] Building 0.1s (9/9) FINISHED docker:default
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 812B 0.0s
=> [internal] load metadata for docker.io/kitware/trame:1.2-glvnd-runtim 0.0s
=> [1/4] FROM docker.io/kitware/trame:1.2-glvnd-runtime-ubuntu20.04-py39 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 258B 0.0s
=> CACHED [2/4] RUN install -d -o trame-user -g trame-user /deploy 0.0s
=> CACHED [3/4] RUN mkdir -p /opt/paraview && cd /opt/paraview && wget - 0.0s
=> CACHED [4/4] COPY --chown=trame-user:trame-user ./server /deploy/serv 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:f1fab77dfe026e4c1e200b89c0317b23b224b7a02090f 0.0s
=> => naming to docker.io/library/pv-visualizer 0.0s
When I run the script ./scripts/run_image.sh, I got following error
➜ docker git:(master) ✗ docker run --rm --gpus all -p 8080:80 -v "$HOME:/data" pv-visualizer
* Restarting Apache httpd web server apache2
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
...done.
Starting server...
/opt/trame/activate_venv.sh: line 1: /deploy/server/venv/bin/activate: No such file or directory
/opt/trame/run.sh: line 20: /deploy/server/launcher.json: No such file or directory
Starting the wslink launcher at
/usr/bin/python: Error while finding module specification for 'wslink.launcher' (ModuleNotFoundError: No module named 'wslink')
It seems you did not build your app /deploy/server/venv/bin/activate: No such file or directory.
Or at least that step failed.
Can you explain what you did with all the steps? I might be able to spot what you skipped.
Hi Sebastien,
I did follow guide in that Git repo (GitHub - Kitware/paraview-visualizer: Web frontend to ParaView based on trame) . Steps are :
- Clone git :
git clone https://github.com/Kitware/paraview-visualizer.git - Change dir into parview-visualizer/docker :
cd paraview-visualizer/docker - Run script :
./scripts/build_image.sh. I managed to build the image : pv-visualizer
➜ ~ docker images -a | grep pv-visualizer
pv-visualizer latest 4c54783983cb 18 hours ago 2.33GB
- Run script:
./scripts/run_image.sh. Since this script wrappers docker run so there is no error appeared in the terminal. I did run docker directly by the following command
➜ docker git:(master) ✗ docker run --rm --gpus all -p 8080:80 -v "$HOME:/data" pv-visualizer
* Restarting Apache httpd web server apache2
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.3. Set the 'ServerName' directive globally to suppress this message
...done.
Starting server...
Starting the wslink launcher at
/deploy/server/venv/bin/python: Error while finding module specification for 'wslink.launcher' (ModuleNotFoundError: No module named 'wslink')
The build_server.sh step is failing. You need to know why.
You may have to delete the generated server directory to fully re-run that step. Then report back on any log output and content of that server directory.
Here is the build_image run after I delete generated server directory
➜ docker git:(master) ✗ ./scripts/build_image.sh
~/workingspace/paraview-visualizer/docker ~/workingspace/paraview-visualizer/docker
~/workingspace/paraview-visualizer/docker
* Restarting Apache httpd web server apache2
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
...done.
Starting server...
/opt/trame/activate_venv.sh: line 1: /deploy/server/venv/bin/activate: No such file or directory
/opt/trame/run.sh: line 20: /deploy/server/launcher.json: No such file or directory
Starting the wslink launcher at
/usr/bin/python: Error while finding module specification for 'wslink.launcher' (ModuleNotFoundError: No module named 'wslink')
[+] Building 0.1s (11/11) FINISHED docker:default
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 742B 0.0s
=> [internal] load metadata for docker.io/kitware/trame:glvnd_2023-08 0.0s
=> [1/6] FROM docker.io/kitware/trame:glvnd_2023-08 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 684B 0.0s
=> CACHED [2/6] RUN install -d -o trame-user -g trame-user /deploy 0.0s
=> CACHED [3/6] COPY --chown=trame-user:trame-user . /deploy 0.0s
=> CACHED [4/6] RUN /opt/trame/entrypoint.sh build 0.0s
=> CACHED [5/6] RUN mkdir -p /opt/paraview && cd /opt/paraview && wget -qO- https://www.paraview.org/files/v5.10/ParaView-5.10.0-egl-MPI-Linux-Python3.9-x86_64.tar.gz | tar --strip-components=1 -xzv 0.0s
=> CACHED [6/6] COPY --chown=trame-user:trame-user ./server /deploy/server 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:4c54783983cb2a2fd1cc4f90680712806942906a8018d08d1bf65cfae3197bf2 0.0s
=> => naming to docker.io/library/pv-visualizer 0.0s
Not sure why /deploy/server/venv/bin/activate is not mounted by the original image kitware/trame:glvnd. Here is my Dockerfile in paraview-visualizer/docker folder
➜ docker git:(master) ✗ cat Dockerfile
FROM kitware/trame:glvnd_2023-08
# Create the /deploy directory with trame-user as the owner
RUN install -d -o trame-user -g trame-user /deploy
COPY --chown=trame-user:trame-user . /deploy
RUN /opt/trame/entrypoint.sh build
# Install ParaView-EGL
ARG PV_URL=https://www.paraview.org/files/v5.10/ParaView-5.10.0-egl-MPI-Linux-Python3.9-x86_64.tar.gz
#ARG PV_URL=https://www.paraview.org/files/nightly/ParaView-master-5.10.1-1278-g070ee00d18-egl-MPI-Linux-Python3.9-x86_64.tar.gz
RUN mkdir -p /opt/paraview && cd /opt/paraview && wget -qO- $PV_URL | tar --strip-components=1 -xzv
ENV TRAME_PARAVIEW=/opt/paraview
# Copy the server directory
COPY --chown=trame-user:trame-user ./server /deploy/server
Can you just run build_server.sh since that is the step that is failing. The build_image.sh is supposed to call that build_server.sh script, but from the output you are sharing, I don’t see any of that.
Hi Sebastien,
This is what build_image.sh prints in console
➜ docker git:(master) ✗ ./scripts/build_server.sh
~/workingspace/paraview-visualizer/docker ~/workingspace/paraview-visualizer/docker
~/workingspace/paraview-visualizer/docker
* Restarting Apache httpd web server apache2
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
...done.
Starting server...
/opt/trame/activate_venv.sh: line 1: /deploy/server/venv/bin/activate: No such file or directory
/opt/trame/run.sh: line 20: /deploy/server/launcher.json: No such file or directory
Starting the wslink launcher at
/usr/bin/python: Error while finding module specification for 'wslink.launcher' (ModuleNotFoundError: No module named 'wslink')
It seems that the build step is skipped probably because of the presence of the server directory from a previous failing build step. So still no information of why the build step fail(ed).
Hi Sebastien,
Thanks a lot for your carefully looking into it. No worry, I’ve managed to build my own docker app with gpu enabled. I have another question : application on server side takes a lot of RAM from GPU. When you deploy your trame app to cloud environment, how do you deal with that ? Do you consider that your trame app will blow out the GPU RAM ?
I guess it really depend on what you are loading and visualizing. But if you can not share a machine across users, you probably need a setup similar to JupyterHub where you start your new machine dynamically for your app. Another path could be to leverage Trame reverse websocket connection, so you could start a job “as a lambda” and have a central/static routing service. The latter is what we do when we schedule jobs on HPC.
Hi Sebastien,
My app is about loading DICOM files and viewing 3D images. I am using Trame reverse websocket. Where each connection coming in will be creating a new python job. But this way consumes a lot of memory (RAM) of GPU. Is there another way to handle it ? Or do you have any plan in the future to improve the wslink framework to handle situation like 1 single application handling multiple connections ?
Just to clarify, you are not using the reverse websocket. You are using the regular 1 process per user with the regular websocket. Unless you’ve create a double sided websocket proxy service.
If your data is shared across users, your would indeed be able to save memory, but otherwise, I don’t see it happening. You can create such application but you will need to make sure each client gets its own view and own name-spaced state.
Normally the 1 server for many users is supported to handle collaboration. But a good software design would allow you to manage several users independently on a single process.
Hi sebastien, can you please help to clarify what reverse websocket mean ? having an apache in the middle between user and python process does not mean reverse websocket ? Can you give me an example how do you set up reverse websocket ?
Reverse websocket is needed when the schedule job is not reachable (network wise) by the main web server. This setup is required on HPC (compute node vs login node) or with lambda like execution (target IP unkown). Then you need a proxy service to properly connect 2 client websocket connection together.
Are you asking me how to schedule a job on HPC and how you would make it work with trame?
Again that is to target 1 (or more) machine(s) per user rather than 1 process per user.
Based on what you said, it seems that you want 1 process for many users.
Hi Sebastien,
Yes what I want is to have 1 process for many users. I have only one dedicated server and I want to serve my 3D application for many users. Do you have any plans for doing this ?
Thanks
Like I said, this is supported but fall onto the design of your application. Trame by default target the process isolation. If you want to break that isolation because you want to reuse loaded datasets across users, you will need to come up with an architecture that match your workflow. Since I don’t know your flow and how you aim to reuse datasets, I can’t guide you on how you could achieve that.