I know that a lot of people have problems when writing PVTU files using the
vtkXMLPUnstructuredGridWriter object due to the fact that each processor writes a VTU file per PVTU output file. This means that for simulations with 1000 processors, each PVTU would be linked to 1000 vtu files. This doesn’t scale well and as a lot of clusters impose a limit on the amount of files per user. Therefore, parallel simulation codes can’t generally rely on the
vtkXMLPUnstructuredGridWriter to write the data of their simulations. In our case we use
vtkPvoWriter which is an external vtk library found in Github to reduce the amount of VTU files per PVTU.
However, I was wondering if we could use a more vtkonic way of dealing with this issue. I have created a filter which reduces the amount of VTU files linked to a PVTU by selecting root processors (based on the number of VTU files to be output) and by gathering/reducing the
vtkUnstructuredGrid objects of the neighbor ranks and splitting the MPI communicator before writing the PVTU. This allows to reduce the number of files per PVTU at the expense of increasing the memory footprint due to the gathering/reduction of the
vtkUnstructuredGrid objects of the neighbor ranks.
I was wondering two things:
- Is VTK interested in adding this filter to their Parallel Write objects?
- In case 1) is true, which is the effort needed to merge this write to VTK’s repo? I suppose that there should be a Test associated to the filter, certain clang format, etc.