surface reconstruction in javascript

Web
1

Hi,
I’m visualizing a point cloud in browser to user, along with a set of controllers that by interacting with them, point cloud gets updated and visualized again in real time.
Now I want to visualize the surface instead of point cloud. It means by interaction of user, the new point cloud gets generated, its surface gets constructed and visualized in the realtime.

But seems like there’s no VTK.js implementation for surface reconstruction.

for such scenario what kind of solution you’ll make:
1-sending point cloud to server every time the user changes it interactively, then constructing surface on server using vtk in python, sending back to client and visualizing it
2-implementing a lightweight surface reconstrcution algorithm in javasctipt ( I’m not sure how convenient it is to do?) or maybe you can let me know of any libraries out there that does the surface reconstruction in js.
3-using web assembly or cross-compilataion schemes to port surface reconstruction from an existing vtk implementation to js. (I have no familiarity how to do that, but willing to learn if that’s not a real tricky job and there’s some resources how to do that)

I appreciate your solutions and ideas,
Best Regards,
Saeed

2

It will depend on the amount of data (number of points) and speed that you need to achieve.
Your options seems reasonable but it is hard to pick one without more knowledge of the data and/or doing some experiments.

1 Like
3

@Sebastien_Jourdain Well, actually it’s desirable that users can see no lags or delays while they’re interacting with the controllers that update the point cloud. for now the point cloud has points in the order of 3000 points.

Considering these setup and expectations, I’m wondering which option would be a better choice?

Best regards,
Saeed

4

Where are the point coordinates coming from? From a web service, from a client lidar device?

3000 points is fairly small. So it might be fine to process it all on the client side, but you will have to validate that assumption.

If you have the data remotely, I would go with a vtkweb/pvweb setup where you synch the scene with vtk.js on the client side while letting all the computation happen on the server side.

1 Like
5

Well, all the coordinates are calculated in the client side.
Actually, I have a few csv files containing some parameters in the form of matrices and vectors. In the beginning, these parameters are loaded into the window, and during the interaction of the user with the controllers on the page, the coordinates of point cloud gets calculated in real-time based on the parameters and the controller values changed by user, on any change (this can be really frequently).
so no data is remote and it’s all computed on the page.
for now all is going well and the real-time calculation of point cloud is really smoothly possible on the client, yet I’m not sure about the overhead of surface reconstruction after each point cloud generation.

6

I would try to run the surface reconstruction in a worker to see how it behave assuming you can implement it in JS or WASM.

7

I agree. So for running a worker thread with a surface_reconstruction functionality I need first to port surface reconstruction from C++ to WASM.
@Sebastien_Jourdain Is there a good tutorial or comprehensive example on how to do that (especifically in vtk code, like the one that I want to do)?

8

Not really, but this would be the best entry point.

2 Likes
9

I agree with Sebastien concerning the link he referred to. I am currently running VTK in WASM, including dynamically creating polydata and imagedata, and the speed seems to be great. I used the link Sebastien referred to and was up and running in a relatively short time.

1 Like
10

The MultiCone Emscripten example is what I based my code on since it shows interaction between javascript and wasm.

1 Like
11

Thank you for the link, I’ve been able to build WebAssembly examples (Cone and MultiCone) with Emscripten Docker.
Now I’m aiming for building vtkSurfaceReconstructin to WASM and use it along with my JS app, seems like it’s not working the same way that I did with examples above (I guess CMakeLists should be changed in some way, I tried some but not working yet).
do you have any points or resources on how to continue?

12

It should but right now I don’t have anything more than what I shared earlier.

13

Make a copy of the MultiCone folder in the src\Examples\Emscripten\Cxx directory and name the folder whatever your project is (keep your copy in the src\Examples\Emscripten\Cxx directory) . Then modify the CMakeList.txt file in the new folder to match your project sources. Here is my CMakeList.txt file for my project named SwisRadar:

cmake_minimum_required(VERSION 3.13)
project(SwisRadar)

# -----------------------------------------------------------------------------
# EMSCRIPTEN only
# -----------------------------------------------------------------------------

if (NOT EMSCRIPTEN)
  message("Skipping example: This needs to run inside an Emscripten build environment")
  return ()
endif ()

# -----------------------------------------------------------------------------
# Handle VTK dependency
# -----------------------------------------------------------------------------

find_package(VTK
  COMPONENTS
    FiltersSources      # VTK pipeline
    InteractionStyle    # Mouse handling
    RenderingOpenGL2    # For Rendering
    RenderingUI         # For SDL2 Window
)

if (NOT VTK_FOUND)
  message("Skipping example: ${VTK_NOT_FOUND_MESSAGE}")
  return ()
endif ()


# -----------------------------------------------------------------------------
# WebAssembly build options
# (-s) https://github.com/emscripten-core/emscripten/blob/master/src/settings.js
# -----------------------------------------------------------------------------

set(emscripten_options)
list(APPEND emscripten_options
  "--bind"
  --preload-file Data
  "-g3"
  "-O3"
  "SHELL:-s EXPORT_NAME=vtkRadar"
  "SHELL:-s MODULARIZE=1"
  "SHELL:-s ALLOW_MEMORY_GROWTH=1"
  "SHELL:-s DEMANGLE_SUPPORT=1"
  "SHELL:-s EMULATE_FUNCTION_POINTER_CASTS=0"
  "SHELL:-s ERROR_ON_UNDEFINED_SYMBOLS=0"
  "SHELL:-s USE_PTHREADS=0"
  "SHELL:-s WASM=1"  
  "SHELL:-s EXPORTED_FUNCTIONS=\"['_main', '_stop', '_addRadarCue', '_buildScene', '_gotoNextCue', '_gotoPreviousCue', '_gotoFirstCue', '_gotoLastCue']\""
  "SHELL:-s EXTRA_EXPORTED_RUNTIME_METHODS=\"['ccall', 'cwrap']\""
  "SHELL:-s ENVIRONMENT=web"
  # "SHELL:-s ASSERTIONS=2"
  "SHELL:-s USE_BZIP2=1"
  "SHELL:-s FETCH=1"
  
)

# -----------------------------------------------------------------------------
# EXPORTED_FUNCTIONS=\"['_main', '_stop', '_addRadarCue', '_buildScene', '_gotoNextCue', '_gotoPreviousCue', '_gotoFirstCue', '_gotoLastCue']\""
# Compile example code
# -----------------------------------------------------------------------------

add_executable(SwisRadar SwisRadar.cxx SwisVtkRadarWindow.cxx SwisVtkGlobeRenderer.cxx SwisCapRenderer.cxx SwisLineCapActor.cxx SwisPolyCapActor.cxx SwisVtkMapRenderer.cxx SwisVtkRadarLegendRenderer.cxx SwisVtkRpgSuperResRefRenderer.cxx SwisVtkRpgSuperResVelRenderer.cxx SwisVtkScalarBarRenderer.cxx SwisRpgProdLoad.cxx)

target_link_libraries(SwisRadar
  PRIVATE
    VTK::FiltersSources
    VTK::InteractionStyle
    VTK::RenderingOpenGL2
    VTK::RenderingUI
    VTK::RenderingContext2D
    VTK::RenderingAnnotation
    VTK::RenderingVolumeOpenGL2
    VTK::RenderingVolume
    VTK::IOImage
    VTK::IOXML
    VTK::InfovisLayout
    VTK::GeovisCore
    VTK::ChartsCore
    VTK::IOLegacy
)

target_compile_options(SwisRadar
  PUBLIC
    ${emscripten_options}
)

target_link_options(SwisRadar
  PUBLIC
    ${emscripten_options}
)

# -----------------------------------------------------------------------------
# VTK modules initialization
# -----------------------------------------------------------------------------

vtk_module_autoinit(
  TARGETS  SwisRadar
  MODULES  ${VTK_LIBRARIES}
)

# -----------------------------------------------------------------------------
# Copy HTML to build directory
# -----------------------------------------------------------------------------

add_custom_command(
  TARGET SwisRadar
  COMMAND
    ${CMAKE_COMMAND} -E copy_if_different
      "${CMAKE_CURRENT_SOURCE_DIR}/SwisRadar.html"
      $<TARGET_FILE_DIR:SwisRadar>
)

Then I create a project folder in the directory: “build-examples-wasm\Emscripten\Cxx” and navigate to that folder. Then run the following command (changing the name to your project):

cmake \
  -G Ninja \
  -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE} \
  -DVTK_DIR=/work/build-vtk-wasm \
  /work/src/Examples/Emscripten/Cxx/SwisRadar

And then run cmake --build .

1 Like
14

Thank you @Donny_Zimmerman,
so this is the state now:
I had js1, created in vtk.js built by “npx webpack” and is responsible for dynamically regenerating the point cloud coordinates on every interaction of the user and creating the polydata out of it.
now I have a js2+wasm created in c++ and built with emscripten, from which I need a kind of api capability of receiving a polydata and returning a surface. I saw in MultiCone example that is creating APIs and I also did it. (I made a function that by receiving a polydata returns a surface and it’s exported in js2+wasm as well).

so now I want js1 to talk to js2, sending polydata and receiving surface. how is that possible? or am I getting something wrong?

So to put it in anothe way, getting back to my first question: I need surface reconstruction capability to be added to the js program that is working with point clouds now.
My thought was that for example if I need the ability of vtkSurfaceReconstructionFilter.cxx in my code, I can compile this file (vtkSurfaceReconstructionFilter.cxx) to a wasm and use it like a library as I was doing in c++ or python, but as @Donny_Zimmerman showed, It’s like I should make an example program (like cone, or MultiCone) in c++ that uses vtkSurfaceReconstructionFilter and then build that program to a wasm and js. but then how should I glue that to my previous program?

I appreciate if anybody elaborate a bit on that.

15

Let me ask the question in broader sense, that can help others and clears things out for me:

let’s say that in a web application I need the functionality already implemented in vtkModuleA.cxx (in this case vtkSurfaceReconstructionFilter.cxx under Imaging/Hybrid). what’s the way for doing that:
-building vtkModueA into wasm and using it?
-or writing an example program like exampleA.cxx that uses vtkModuleA, and building exampleA into Wasm and talking to vtkModuleA trhough exampleA?

@Sebastien_Jourdain mentioned running surface reconstruction in a worker, then what message am I supposed to send and received from this worker?
like sending a vtkpolydata and receiving a vtksurface? then how these objects are serialized to be sent and received?

Having an answer to this really clears things out.
thank you so much in advance

16

For my project I do all VTK related functions in WASM, including rendering. The only thing I pass between WASM and JS are simple strings and numbers. I think it gets much more complicated when you want to pass more complex objects. I think it is because WASM and JS don’t share the same memory space. I think @Sebastien_Jourdain was working on an example to pass complex data back and forth.

I guess I would ask, why don’t you do all VTK functions in WASM. I am even fetching data from a server in WASM using the Emscripten fetch functionality. The only thing I am passing between WASM and JS are user selection information (not interaction), like go to next frame, etc…

1 Like
17

@Donny_Zimmerman you mentioned that you do all VTK-related functionalities in WASM. that’s super interesting. So how you mean doing them all in WASM? because I’m not doing anything with WASM, except the WASM file is generated file when I build cxx file with Emscripten. So how can I do them all? you mean in c++ program connect to the server and read the data? so it’s like implementing the whole web app functionality in c++?

regarding your question, Well, actually I was having two files because in the first one I was using VTK.js and so building it with npm every time, to work with the generated app.js in my page. In the other js file I was trying to use VTK functionality that’s not in VTK.js, by building those functionalities from VTK c++ using Emscripten.

thank you, now I combined the two js files into one. So now I’m:
1-building the a.cxx (VTK app) into a.js and a.wasm using Emscripten
2-I add a.js content to the b.js in which I’m using VTK.js functionalities
3-build b.js using NPM into app.js
4-use the resulting app.js in my application

So I made a function in a.cxx that taking a polydata, returns a surface. but after the first step I’m not sure how to access and use that function in b.js. that’s what is not clear to me and I’m trying to see where I should restructure. then maybe the answer is in your next comment, knowing how you do almost everything in WASM.
So, looking forward to that.

18

@Saeed I am doing all of the data fetching, VTK filtering, and VTK rendering with C++ compiled to WebAssembly (WASM) using Emscripten. The only functionality that is done in JS is simple user interaction, such as selecting parameters for what data the user wants to see. This is passed as a string from JS to WebAssembly and in WebAssembly I am fetching the data based on the string that was passed. I then display the data using VTK C++.

So basically, the JS handles all button and menu selections only. I am not using vtk.js at all. This information is passed to WebAssembly as a simple string, and then in C++ I am fetching the data and displaying it with VTK C++.

Look up “Emscripten Fetch” to see how to “fetch” data from a server.

1 Like
19

@Donny_Zimmerman Thank you,
I tried to use the synchronous Emscripten fetch code copied from Emscripten API ref and Emscripten build options like the one in CMakeLists you sent in your comment, but it returns null in fetch->data, while asynchronous fetching works.
I searched on the web and seems like deadend discussions on that in stackoverflow and github repo of Emscripten. so did you use Synchronous method and that worked properly? any tricks?

20

I am using async fetch methods. I was trying to use sync methods, but was running into the same issues you are. There are conflicting flags between the Emscripten documentation and what the errors state when actually setting those flags. I haven’t had time to pursue the errors.