WASM Regression tests (file locations)

Development
3

Hello @Jens_Munk_Hansen

I was out of office for the last week.

Emscripten provides a default memory file system backend MEMFS. With that, all file I/O from C++ assume a virtual filesystem in memory mounted at /. For VTK, we run regression tests on each MR and nightly using chrome-for-testing browser with extra plumbing in the VTK test utilities module.

With regards to test data, some unit tests want to read a data file or a baseline image and later write a difference image. So the infrastructure is capable of handling both reads and writes.

Approach 1: Using VTK testing infrastructure

If you’re writing VTK based unit tests using the vtkModuleTesting - VTK documentation cmake functions, simply add these two lines in your test cxx and the below lines in your cmake right before you call vtk_add_test_cxx and define the cmake variable VTK_TESTING_WASM_ENGINE to the path to a browser executable.

The vtkModuleTesting.cmake does all the magic of serving the wasm, and running a custom python server which implements preloading and dumping files. Just run ctest and you will see browser windows opening and closing each running your unit test.

#include "vtkRegressionTestImage.h"
...
...
int TestBar(int argc, char* argv)
{
 ...
 ...
 vtkRegressionTestImage(renderWindow); // see Testing/Rendering/vtkRegressionTestImage.h for more macros.
}

if (CMAKE_SYSTEM_NAME STREQUAL "Emscripten")
  set(_vtk_test_cxx_wasm_enabled_in_browser ON)
  set(_vtk_test_emscripten_extra_linker_args
    "-sASSERTIONS=1"
    # O2 or higher make wasm-opt terribly slow.
    # test executables take a minute to link.
    # we need emscripten to emit code that handles exit code and atexit(s)
    "-sEXIT_RUNTIME=1"
    # tests dynamically allocate memory, can easily go over the preset limit.
    "-sALLOW_MEMORY_GROWTH=1")
endif ()

vtk_add_test_cxx(MyTestExecutableName tests
  TestFoo.cxx
  TestBar.cxx
)

Approach 2: Doing it yourself

The VTK Emscripten test utilities uses a custom protocol to communicate requests to preload or dump a file from the wasm client to the server hosting the wasm module. If you want to use a node js server instead of a python server like we do in VTK, implement these routes in your node server.

Routes:

  1. /preload?file=full_path_to_file: wasm unit test makes a GET with the full path of a file in its query parameters. The server should respond with the contents of the file or 404 if a file does not exist
  2. /dump?file=full_path_to_file: wasm unit test initiates a POST with the full path of a file in its query parameters and sends the entire contents of the output file in the POST content. “Content-Length” is the bytesize of file.

Reference Client-side implementation in VTK:

  1. vtkEmscriptenTestUtilities.h
  2. vtkEmscriptenTestUtilities.cxx
  3. vtkEmscriptenTestUtilities.js

Reference Server-side implementation in VTK:

  1. The vtkTestHTTPHandler.translate_path method implements the /preload route. This responds to requests for a file.
  2. The vtkTestHTTPHandler.do_POST method implements routes like /dump, /console_output and /exit. These are triggerred when the wasm application wants to write a file to disk through fwrite, print messages to cout/cerr and exit(code) respectively.