vtkMultiBlockDataSet equivalent for VTKHDF files

Support
1

I’m using .vtm files to handle results from simulations where singe timestep output comes in several file formats (.vtp, .vtu and sometimes .vti). Gathering all of the data in a .vtm file makes loading into Paraview very easy, particularly with the MultiBlockInspector and if you have multiple meshes in various groups, like the simplified example below.

<VTKFile type="vtkMultiBlockDataSet" version="1.0" byte_order="LittleEndian">
  <vtkMultiBlockDataSet>
    <Block index="0" name="Base">
      <Piece index="0">
        <DataSet index="0" file="cube_base.vtp"/>
      </Piece>
    </Block>
    <Block index="1" name="Sides">
      <Piece index="0">
        <DataSet index="0" file="cube_back.vtp"/>
      </Piece>
      <Piece index="1">
        <DataSet index="1" file="cube_left.vtp"/>
      </Piece>
      <Piece index="2">
        <DataSet index="2" file="cube_right.vtp"/>
      </Piece>
      <Piece index="3">
        <DataSet index="3" file="cube_front.vtp"/>
      </Piece>
    </Block>
    <Block index="2" name="Top">
      <Piece index="0">
        <DataSet index="0" file="cube_top.vtp"/>
      </Piece>
    </Block>
  </vtkMultiBlockDataSet>
</VTKFile>

I’ve been trying to switch to VTKHDF recently but I haven’t quite figured out how to replicate this functionality in the VTKHDF format. It doesn’t seem possible to be able to add multiple blocks to a single VTKHDF file, which I thought would have been the easiest way. There also doesn’t seem to be support for a multiblock index file like the vtm file.

I’m aware there is some discussion on supporting multiple files for transient data which I would really like to see as all my temporal data files are large and it doesn’t make sense to have HDF5 files that are tens or hundreds of GB in size when the could easily be split - would the ability to do multiple blocks / meshes for a single datasets be possible this way?

2

It is supported.

https://docs.vtk.org/en/latest/design_documents/VTKFileFormats.html#partitioneddatasetcollection-and-multiblockdataset

FYI @lgivord @Louis_Gombert

3

Thanks. I completely missed that section.

Quick question - it says each block should describe a valid VTKHDF root - does that mean that each block for PolyData would follow the layout described here:

image
image1759×838 91.9 KB

Also is there an example file for vtkPartitionedDataSetCollection (PDC) and vtkMultiBlockDataSet (MB) that can be downloaded so that i can better understand?

4

hello @jpmorr,

Quick question - it says each block should describe a valid VTKHDF root - does that mean that each block for PolyData would follow the layout described here:

yes

Also is there an example file for vtkPartitionedDataSetCollection (PDC) and vtkMultiBlockDataSet (MB) that can be downloaded so that i can better understand?

you can check the data used for testing it in VTK

5

@lgivord Thanks. I had a look at downloading some example files that I found here but it’s just a load of sha files.

Where are the actual VTKHDF files stored?

6

I’ve still not figured out how to download the test data files from gitlab but i@ve tried to write avtkPartitionedDataSetCollection without being able to check against an example. While I think I’ve got the format correct, paraview is not able to load it with an error saying no valid files found.

Here’s my file (which is just 4 random polydata blocks): hdf_multiblock_test.vtkhdf (2.4 MB)

Maybe someone could have a look at it and point out what I’ve got wrong - I assume it’s the assembly part.

7

I’ve still not figured out how to download the test data files from gitlab

From the website we can download the .sha512 indeed, if we want directly the data it should be somewhere here I believe: https://www.paraview.org/files/, @mwestphal any inputs?

In your case, I attach the data here :
test_composite.hdf (27.1 KB)

paraview is not able to load it with an error saying no valid files found.

that’s not the error that I have, which version of paraview do you use? On master, I have this issue:

ERROR: In vtkHDFUtilities.txx, line 162
(nullptr): Index attribute not found

The overall structure of your file is correct however you miss the Index attribute for the assembly, you can refer to the second paragraph for more details : VTK File Formats - VTK documentation

8 
sha=`cat VTK/path/to/file.ext.sha512`
wget https://www.vtk.org/files/ExternalData/SHA512/$sha -O file.ext
2 Likes
9

Thanks. I completely missed that index attribute when I was reading.

Can I ask why the index attributes is required, since the assembly data is linked to the block data in hdf5 file?

Also, is the index somewhat arbitrary? Does it matter if the assembly order of the blocks does not match the block index?

10

Can I ask why the index attributes is required, since the assembly data is linked to the block data in hdf5 file?

It’s link to how the vtkPartitionedDataSetCollection works in VTK, as multiple leafs can point to the same dataset, PDC needs a flat structure and this is the purpose of this Index.

Also, is the index somewhat arbitrary?

each index should be unique, I don’t think there is another limitation

Does it matter if the assembly order of the blocks does not match the block index?

It doesn’t matter :slightly_smiling_face:

11

Thanks!!

12

You can also grab all data files from https://www.vtk.org/files/release/9.3/VTKDataFiles-9.3.1.tar.gz. It’s organized into the same directory structure as VTK, so finding data in it should be easy.