If the points are duplicate over multiple processes, one has to adjust the global indices on all other processes, which makes it non-trivial (i.e. requires a global exscan) to write data in parallel, and would make the data more challenging to read in again at a later stage, as one would have to eliminate duplicate points.
Thus I would go for the non-duplicate version, i.e. I have M global nodes, which in the connectivity array is filled with the global indices. However, then it is unclear to me how to specify that a node is shared between multiple (let say two) processes, as the only information we provide in the VTKHDF file is
We describe the split into partitions using HDF5 datasets
NumberOfConnectivityIds,NumberOfPointsandNumberOfCells.
However, this doesn’t indicate the start and end of NumberOfPoints.
Following is a minimal example of how a mesh is distributed in my code:
We have a 1x1 unit square consisting of two triangles, and in total four nodes.
Each processes owns a single triangle, lets say
Proc 0 owns the cell with vertices [0,0], [1,0], [0,1]
Proc 1 owns the cell with vertices [1,0], [1,1],[0,1]
If we write the unique nodes to file, say in the following order
[[0,0],[1,0],[0,1],[1,1]]
we need to indicate that process 0 has to access the first three points, range [0,3) while process 1 wants to access the nodes in range [1, 4).
I am not sure how to specify this with VTKHDF.
To make the issue even clearer, one could add another process with a single cell
[-1,0], [0,0], [0, 1], where we append [-1,0] at the end of the points array, meaning that process (2) would need point indices 0, 2 and 4 available (as that would be the connectivity array).
Would NumberOfPoints be three for each process?