# WASM Regression tests (file locations)

**URL:** <https://discourse.vtk.org/t/wasm-regression-tests-file-locations/15040>\
**Category:** Development\
**Created:** [December 17, 2024, 8:37pm UTC](https://discourse.vtk.org/t/wasm-regression-tests-file-locations/15040 "2024-12-17T20:37:17Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jens\_Munk\_Hansen](https://discourse.vtk.org/user_avatar/discourse.vtk.org/jens_munk_hansen/32/1215_2.png) [@Jens\_Munk\_Hansen](https://discourse.vtk.org/u/Jens_Munk_Hansen)\
**Post date:** [December 17, 2024, 8:37pm UTC](https://discourse.vtk.org/t/wasm-regression-tests-file-locations/15040/1 "2024-12-17T20:37:17Z")

</div>

I have some experiments with regards to regression tests for WASM. I do the following

1. Use ExternalData\_add\_test and add\_test
2. Autogenerate a node.js script, which can copy data into WASM filesystem and call a function `TestXXX(int argc, char* argv)`, where I files can be placed in the file system
3. The output are written to WASM file system
4. The node.js script can copy them to output

One issue. Many of our tests, just take a folder and the filename is written inside the `.cxx` file.

First, I will get it up and running for a test, where the input file and output files are specified in the CMake system. Do we have such a test? If yes, can you provide a good example?

Would it be an idea to always keep input and baseline in the CMake system and not written in `.cxx`?

I realized there is a bug in Emscripten about reading from stdio using a stream. It is though easy to accept a fixed string and execute tests like, `node vtkCommonCoreCxxTests TestSomething`.

---

<div class="post-metadata">

**Author:** ![Jens\_Munk\_Hansen](https://discourse.vtk.org/user_avatar/discourse.vtk.org/jens_munk_hansen/32/1215_2.png) [@Jens\_Munk\_Hansen](https://discourse.vtk.org/u/Jens_Munk_Hansen)\
**Post date:** [December 20, 2024, 11:57am UTC](https://discourse.vtk.org/t/wasm-regression-tests-file-locations/15040/2 "2024-12-20T11:57:47Z")

</div>

I figure out that you have made test using preload of data in the browser. That is a different approach. Any change that we will get this to work also from Node? Would be nice with a simple example, then developers can follow and help on getting WASM tests to work

---

<div class="post-metadata">

**Author:** ![jaswantp](https://discourse.vtk.org/user_avatar/discourse.vtk.org/jaswantp/32/10046_2.png) [@jaswantp](https://discourse.vtk.org/u/jaswantp)\
**Post date:** [December 23, 2024, 2:37pm UTC](https://discourse.vtk.org/t/wasm-regression-tests-file-locations/15040/3 "2024-12-23T14:37:22Z")

</div>

Hello @Jens_Munk_Hansen

I was out of office for the last week.

Emscripten provides a default memory file system backend [MEMFS](https://emscripten.org/docs/api_reference/Filesystem-API.html#file-system-api). With that, all file I/O from C++ assume a virtual filesystem in memory mounted at `/`. For VTK, we run regression tests on each MR and nightly using [chrome-for-testing](https://github.com/GoogleChromeLabs/chrome-for-testing) browser with extra plumbing in the VTK test utilities module.

With regards to test data, some unit tests want to read a data file or a baseline image and later write a difference image. So the infrastructure is capable of handling both reads and writes.

**Approach 1: Using VTK testing infrastructure**

If you’re writing VTK based unit tests using the [vtkModuleTesting - VTK documentation](https://docs.vtk.org/en/latest/api/cmake/vtkModuleTesting.html#c-tests) cmake functions, simply add these two lines in your test cxx and the below lines in your cmake right before you call `vtk_add_test_cxx` and define the cmake variable `VTK_TESTING_WASM_ENGINE` to the path to a browser executable.

The `vtkModuleTesting.cmake` does all the magic of serving the wasm, and running a custom python server which implements preloading and dumping files. Just run `ctest` and you will see browser windows opening and closing each running your unit test.

```cpp
#include "vtkRegressionTestImage.h"
...
...
int TestBar(int argc, char* argv)
{
 ...
 ...
 vtkRegressionTestImage(renderWindow); // see Testing/Rendering/vtkRegressionTestImage.h for more macros.
}

```

```py
if (CMAKE_SYSTEM_NAME STREQUAL "Emscripten")
  set(_vtk_test_cxx_wasm_enabled_in_browser ON)
  set(_vtk_test_emscripten_extra_linker_args
    "-sASSERTIONS=1"
    # O2 or higher make wasm-opt terribly slow.
    # test executables take a minute to link.
    # we need emscripten to emit code that handles exit code and atexit(s)
    "-sEXIT_RUNTIME=1"
    # tests dynamically allocate memory, can easily go over the preset limit.
    "-sALLOW_MEMORY_GROWTH=1")
endif ()

vtk_add_test_cxx(MyTestExecutableName tests
  TestFoo.cxx
  TestBar.cxx
)

```

**Approach 2: Doing it yourself**

The VTK Emscripten test utilities uses a custom protocol to communicate requests to preload or dump a file from the wasm client to the server hosting the wasm module. If you want to use a node js server instead of a python server like we do in VTK, implement these routes in your node server.

_Routes:_

1. `/preload?file=full_path_to_file`: wasm unit test makes a GET with the full path of a file in its query parameters. The server should respond with the contents of the file or 404 if a file does not exist
2. `/dump?file=full_path_to_file`: wasm unit test initiates a POST with the full path of a file in its query parameters and sends the entire contents of the output file in the POST content. “Content-Length” is the bytesize of file.

_Reference Client-side implementation in VTK:_

1. [vtkEmscriptenTestUtilities.h](https://gitlab.kitware.com/vtk/vtk/-/blob/master/Testing/Core/vtkEmscriptenTestUtilities.h)
2. [vtkEmscriptenTestUtilities.cxx](https://gitlab.kitware.com/vtk/vtk/-/blob/master/Testing/Core/vtkEmscriptenTestUtilities.cxx)
3. [vtkEmscriptenTestUtilities.js](https://gitlab.kitware.com/vtk/vtk/-/blob/master/Testing/Core/vtkEmscriptenTestUtilities.js)

_Reference Server-side implementation in VTK:_

1. The [vtkTestHTTPHandler.translate\_path](https://gitlab.kitware.com/vtk/vtk/-/blob/master/Testing/WebAssembly/runner.py#L110) method implements the `/preload` route. This responds to requests for a file.
2. The [vtkTestHTTPHandler.do\_POST](https://gitlab.kitware.com/vtk/vtk/-/blob/master/Testing/WebAssembly/runner.py#L110) method implements routes like `/dump`, `/console_output` and `/exit`. These are triggerred when the wasm application wants to write a file to disk through `fwrite`, print messages to cout/cerr and `exit(code)` respectively.

---

<div class="post-metadata">

**Author:** ![jaswantp](https://discourse.vtk.org/user_avatar/discourse.vtk.org/jaswantp/32/10046_2.png) [@jaswantp](https://discourse.vtk.org/u/jaswantp)\
**Post date:** [December 23, 2024, 9:21pm UTC](https://discourse.vtk.org/t/wasm-regression-tests-file-locations/15040/4 "2024-12-23T21:21:05Z")

</div>

While the current approach in VTK works great for most of the unit tests, there is room for improvement.

Composite dataset readers for ‘.vtm’ or ‘.vtpc’ files indirectly load other ‘.vtp’ files which are not intercepted by the `vtkEmscriptenTestUtilities` APIs I mentioned. Ideally, when running tests that access/write data within the browser, you’d want to intercept all C `fopen`, `fread`, `fclose` calls to basically request the server to send the file over network, preload it in the virtual file system, read it, popluate the buffer with the file contents and return an appropriate status code and similarly intercept `fwrite`.

Emscripten offers a [synchronous virtual XHR backed file system](https://emscripten.org/docs/porting/files/Synchronous-Virtual-XHR-Backed-File-System-Usage.html). The drawback is it requires you to know the exact filename at compile time which is very limiting…

---

<div class="post-metadata">

**Author:** ![Jens\_Munk\_Hansen](https://discourse.vtk.org/user_avatar/discourse.vtk.org/jens_munk_hansen/32/1215_2.png) [@Jens\_Munk\_Hansen](https://discourse.vtk.org/u/Jens_Munk_Hansen)\
**Post date:** [December 26, 2024, 2:30pm UTC](https://discourse.vtk.org/t/wasm-regression-tests-file-locations/15040/5 "2024-12-26T14:30:42Z")

</div>

Thanks. That is what i figured. Would be nice if all tests were made with macros, such that we from CMake knew which files to use. When running the existing Emscripten tests, my system was hanging. I guess it was due to a missing `VTK_TESTING_WASM_ENGINE`, which I haven’t set.

---

<div class="post-metadata">

**Author:** ![jaswantp](https://discourse.vtk.org/user_avatar/discourse.vtk.org/jaswantp/32/10046_2.png) [@jaswantp](https://discourse.vtk.org/u/jaswantp)\
**Post date:** [December 26, 2024, 2:33pm UTC](https://discourse.vtk.org/t/wasm-regression-tests-file-locations/15040/6 "2024-12-26T14:33:43Z")

</div>

Yes, it is far easier to use the system that VTK uses.

> [@Jens\_Munk\_Hansen](#):
>
> When running the existing Emscripten tests, my system was hanging. I guess it was due to a missing `VTK_TESTING_WASM_ENGINE`, which I haven’t set.

If you ran the test in verbose mode with `ctest -VV ..` you will see a message printed like “VTK\_TESTING\_WASM\_ENGINE is not set. test may appear to hang but it is running a http server at url:port”