Threading WASM. Setting the backend.

Support
1

After getting everything to compile, I tried setting the backend to STDThread (WASM). I got a callstack like this

$std::__2::__throw_system_error(int, char const*) @ system_error.cpp:218
$std::__2::thread::thread<vtk::detail::smp::vtkSMPThreadPool::MakeThread()::$_0, void>(vtk::detail::smp::vtkSMPThreadPool::MakeThread()::$_0&&) @ thread.h:216
$vtk::detail::smp::vtkSMPThreadPool::MakeThread() @ vtkSMPThreadPool.cxx:342
$vtk::detail::smp::vtkSMPThreadPool::vtkSMPThreadPool() @ vtkSMPThreadPool.cxx:222
$vtk::detail::smp::vtkSMPThreadPool::GetInstance() @ vtkSMPThreadPool.cxx:424
$vtk::detail::smp::STDThread::GetThreadId() @ vtkSMPThreadLocalBackend.cxx:25
$vtk::detail::smp::STDThread::ThreadSpecific::GetStorage() @ vtkSMPThreadLocalBackend.cxx:177
$vtk::detail::smp::vtkSMPThreadLocalImpl<(vtk::detail::smp::BackendType)1, unsigned char>::Local() @ vtkSMPThreadLocalImpl.h:53
$vtk::detail::smp::vtkSMPThreadLocalAPI<unsigned char>::Local() @ vtkSMPThreadLocalAPI.h:103
$vtkSMPThreadLocal<unsigned char>::Local() @ vtkSMPThreadLocal.h:102
$vtk::detail::smp::vtkSMPTools_FunctorInternal<(anonymous namespace)::FindMaxCell, true>::Execute(int, int) @ vtkSMPTools.h:108
$void vtk::detail::smp::vtkSMPToolsImpl<(vtk::detail::smp::BackendType)1>::For<vtk::detail::smp::vtkSMPTools_FunctorInternal<(anonymous namespace)::FindMaxCell, true>>(int, int, int, vtk::detail::smp::vtkSMPTools_FunctorInternal<(anonymous namespace)::FindMaxCell, true>&) @ vtkSMPToolsImpl.txx:39
$void vtk::detail::smp::vtkSMPToolsAPI::For<vtk::detail::smp::vtkSMPTools_FunctorInternal<(anonymous namespace)::FindMaxCell, true>>(int, int, int, vtk::detail::smp::vtkSMPTools_FunctorInternal<(anonymous namespace)::FindMaxCell, true>&) @ vtkSMPToolsAPI.h:105
$vtk::detail::smp::vtkSMPTools_FunctorInternal<(anonymous namespace)::FindMaxCell, true>::For(int, int, int) @ vtkSMPTools.h:119
$void vtkSMPTools::For<(anonymous namespace)::FindMaxCell>(int, int, int, (anonymous namespace)::FindMaxCell&) @ vtkSMPTools.h:234
$vtkCellArray::GetMaxCellSize() @ vtkCellArray.cxx:1113
$vtkPolyData::GetMaxCellSize() @ vtkPolyData.cxx:596
$vtkCleanPolyData::RequestData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) @ vtkCleanPolyData.cxx:167
$vtkPolyDataAlgorithm::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) @ vtkPolyDataAlgorithm.cxx:76
$vtkExecutive::CallAlgorithm(vtkInformation*, int, vtkInformationVector**, vtkInformationVector*) @ vtkExecutive.cxx:723
$vtkDemandDrivenPipeline::ExecuteData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) @ vtkDemandDrivenPipeline.cxx:450
$vtkCompositeDataPipeline::ExecuteData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) @ vtkCompositeDataPipeline.cxx:151
$vtkDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) @ vtkDemandDrivenPipeline.cxx:249
$vtkStreamingDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) @ vtkStreamingDemandDrivenPipeline.cxx:336
$vtkDemandDrivenPipeline::UpdateData(int) @ vtkDemandDrivenPipeline.cxx:407
$vtkStreamingDemandDrivenPipeline::Update(int, vtkInformationVector*) @ vtkStreamingDemandDrivenPipeline.cxx:426
$vtkStreamingDemandDrivenPipeline::Update(int) @ vtkStreamingDemandDrivenPipeline.cxx:389
$vtkAlgorithm::Update(int) @ vtkAlgorithm.cxx:1523
$vtkAlgorithm::Update() @ vtkAlgorithm.cxx:1517
$vtkImplicitPolyDataDistance::SetInput(vtkPolyData*) @ vtkImplicitPolyDataDistance.cxx:46
$vtkDistancePolyDataFilter::GetPolyDataDistance(vtkPolyData*, vtkPolyData*) @ vtkDistancePolyDataFilter.cxx:84
$vtkDistancePolyDataFilter::RequestData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) @ vtkDistancePolyDataFilter.cxx:53
$vtkPolyDataAlgorithm::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) @ vtkPolyDataAlgorithm.cxx:76

Could it be that I should set VTK_SMP_IMPLEMENTATION_TYPE to STDThread or something? I got the same backtrace. I compiled my WASM witth -sPTHREAD_POOL_SIZE=16, where 16 is my current hardware concurrency. I will look into how the new threads are created - perhaps they try to set affinity (that will for crash)

Enable exceptions, I now see.

Uncaught std::__2::system_error: thread constructor failed: Resource temporarily unavailable

It could seem like all theads are already created and VTK cannot create more threads.

2

I think I figured it out. I managed to change the generated .js file to expose PThread and do a pre-warm of the threads.

I now see a backend STDThread and it says it has 16 threads. I tried using a filter, which supports SMP, but the performance is the same as before. I am using an ES6 module without a main. Earlier (a year ago), I made a small example with a main and I am pretty sure I got a better performance with SMP.

Also, I figured out that the number of threads must match the hardware concurrency (at least with new backend). Different threads execute when using SMPTools, but I don’t our buffers match correctly to SharedArrayBuffers. At least for all the filters that I have tried, I get zero performance.

3

I have made pull request, which may fix this issue. -sPTHREAD_POOL_SIZE_STRICT is apparently not working. Maximum number of threads is still equal to hardware concurrency. What worked was to use EM_ASM_INT. It relies on the fact that no other threads are spawned. https://gitlab.kitware.com/vtk/vtk/-/merge_requests/11749/diffs

Correction, it may work. Only tested in a small C program to see I for sure got the number of threads in my pool.

Damn it. It returns 0, when executed in VTK. I have small ES6 module, when it returns 4 if I set the -sPTHREAD_POOL_SIZE=4.

It occurs since the module is not initialized at this point. Perhaps, I could expose a method for setting the maximum number of threads, like we had in the good old days.

If so, could you point me in the direction of how. Should I make it on the general thread pool and ignore it if the backend is different from STDThread? @jaswantp