I am attempting to use the VTKHDF format to store time series data of an unstructured grid.
So far I have been able to save single timesteps into the hdf file and load them into paraview, but I can’t find any documentation on how to include timeseries data in the hdf file.
I assume it is possible, due to this previous question: Transient vtkHDFReader off by a step
Hi @lucasbanting,
This is a brand new feature that was just integrated in this merge request: https://gitlab.kitware.com/vtk/vtk/-/merge_requests/10094.
The formal reader documentation is forthcoming and we have not yet had the time to put publish it. I’ll be sure to update this thread when it is. Navigating to the merge request linked above, you should be able to find a good deal of information regarding how time is formatted.
Here is also a test data set that you can use for inspiration:
transient_sphere.hdf (82.3 KB)
Hope that helps!
Best regards,
Julien
Also worth mentioning that you can still use paraview to visualize transient vtkhdf files, but I don’t believe that this MR has made it through to an official release yet, so you’ll need a recent nightly build.
What should I use for the offsets when I want to use the same geometry for all time steps?
This is what I tried (I only have CellData):
# offsets that are just all zeros
Steps["PartOffsets"] = zeros(Int64, NSteps)
Steps["PointOffsets"] = zeros(Int64, NSteps)
Steps["CellOffsets"] = zeros(Int64, 1, NSteps)
Steps["ConnectivityIdOffsets"] = zeros(Int64, 1, NSteps,)
Here is my h5dump:
HDF5 "time_series.hdf" {
GROUP "/" {
GROUP "VTKHDF" {
ATTRIBUTE "Type" {
DATATYPE H5T_STRING {
STRSIZE 16;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
DATA {
(0): "UnstructuredGrid"
}
}
ATTRIBUTE "Version" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 2 ) / ( 2 ) }
DATA {
(0): 2, 0
}
}
GROUP "CellData" {
DATASET "E" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 20514816, 3 ) / ( 20514816, 3 ) }
}
}
DATASET "Connectivity" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 213696 ) / ( 213696 ) }
}
DATASET "NumberOfCells" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 1 ) / ( 1 ) }
}
DATASET "NumberOfConnectivityIds" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 1 ) / ( 1 ) }
}
DATASET "NumberOfPoints" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 1 ) / ( 1 ) }
}
DATASET "Offsets" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 71233 ) / ( 71233 ) }
}
DATASET "Points" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 80136, 3 ) / ( 80136, 3 ) }
}
GROUP "Steps" {
ATTRIBUTE "NSteps" {
DATATYPE H5T_STD_I64LE
DATASPACE SCALAR
DATA {
(0): 288
}
}
GROUP "CellDataOffsets" {
DATASET "E" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 288 ) / ( 288 ) }
}
}
DATASET "CellOffsets" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 288, 1 ) / ( 288, 1 ) }
}
DATASET "ConnectivityIdOffsets" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 288, 1 ) / ( 288, 1 ) }
}
DATASET "PartOffsets" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 288 ) / ( 288 ) }
}
DATASET "PointOffsets" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 288 ) / ( 288 ) }
}
DATASET "Values" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 288 ) / ( 288 ) }
}
}
DATASET "Types" {
DATATYPE H5T_STD_U8LE
DATASPACE SIMPLE { ( 71232 ) / ( 71232 ) }
}
}
}
}
All my offsets are zeros, except for CellDataOffsets.
There is a pretty cool example of this made by @berk.geveci for teaching how temporal streaming works. This one of the best resources that I have found about temporal streaming. https://www.kitware.com/developing-hdf5-readers-using-vtkpythonalgorithm/
@jfausty Hi Julien, any idea what this error messae implies when trying to load my hdf file?
ERROR: In vtkExecutive.cxx, line 742
vtkPVCompositeDataPipeline (000001998065FBD0): Algorithm vtkFileSeriesReader (000001998050B010) returned failure for request: vtkInformation (0000019992E723F0)
Debug: Off
Modified Time: 524470
Reference Count: 1
Registered Events: (none)
Request: REQUEST_DATA
FORWARD_DIRECTION: 0
ALGORITHM_AFTER_FORWARD: 1
FROM_OUTPUT_PORT: 0
Hi @lucasbanting,
As is, the excerpt is a little cryptic. It is odd that you are using the vtkFileSeriesReader. Transient vtkHDF shouldn’t be in more than one file.
Can you share the steps to reproduce with the data ?
Best regards,
Julien
Starting from a working VTKHDF unstructured grid file, I added:
For testing, I made all my offsets all zeros just to see if the file would load.
When loading into paraview, I can see the actual time steps in the top bar, but can’t view anything.
I can share my hdf file with you if you’d find it useful.
Do you have a working example with CellData?
Hi @lucasbanting,
What ParaView/VTK version are you using?
ParaView v5.11 and VTK’s release won’t have support for transient data as it is a relatively recent feature. If you are using prebuilt ParaView binaries downloaded off the Kitware website I would advise you to use the nightly packages which are built every day from the master branch.
Best regards,
Julien
I am using 5.11.1-1124 from the website.
Right, can you share your data?
time_series.hdf (498.3 KB)
This is a simple example of something that I think should open, but doesn’t.
If I remove the ‘Steps’ group, it works as a static dataset.
This may very well be a bug that I do not have any more time to investigate.
Please feel free to open an issue at: https://gitlab.kitware.com/vtk/vtk and tag me on it.
Best,
Julien
One subtle thing that I couldn’t find mentioned in the documentation: the NumberOf* datasets in the VTKHDF group must have length number_of_parts * number_of_steps. Since your example only has one part, each of those datasets should have length 18 in your case. Since your mesh isn’t changing, all of the entries should be the same.
That said, when I tried to extend these datasets in your example, it still wasn’t rendered properly in paraview. I’m not sure why that’s the case, but if you want more examples to look at, here’s an hdf file with cell data that I was using to play around with the format. I’m also attaching the python script that generates the file – you can adjust the number of parts there. Ignore the particles function – that was for testing the h5part format, a separate thing.
mytestfile.hdf (138.9 KB)
test.py (8.6 KB)
That worked!
For reference, here is the working h5 file for anyone who stumbles across this:
HDF5 "time_series.hdf" {
GROUP "/" {
GROUP "VTKHDF" {
ATTRIBUTE "Type" {
DATATYPE H5T_STRING {
STRSIZE 16;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
DATA {
(0): "UnstructuredGrid"
}
}
ATTRIBUTE "Version" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 2 ) / ( 2 ) }
DATA {
(0): 2, 0
}
}
GROUP "CellData" {
DATASET "E" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 134928, 3 ) / ( 134928, 3 ) }
}
}
DATASET "Connectivity" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 22488 ) / ( 22488 ) }
}
DATASET "NumberOfCells" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 18 ) / ( 18 ) }
}
DATASET "NumberOfConnectivityIds" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 18 ) / ( 18 ) }
}
DATASET "NumberOfPoints" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 18 ) / ( 18 ) }
}
DATASET "Offsets" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 7497 ) / ( 7497 ) }
}
DATASET "Points" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 22488, 3 ) / ( 22488, 3 ) }
}
GROUP "Steps" {
ATTRIBUTE "NSteps" {
DATATYPE H5T_STD_I64LE
DATASPACE SCALAR
DATA {
(0): 18
}
}
GROUP "CellDataOffsets" {
DATASET "E" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 18 ) / ( 18 ) }
}
}
DATASET "CellOffsets" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 18, 1 ) / ( 18, 1 ) }
}
DATASET "ConnectivityIdOffsets" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 18, 1 ) / ( 18, 1 ) }
}
DATASET "PartOffsets" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 18 ) / ( 18 ) }
}
DATASET "PointOffsets" {
DATATYPE H5T_STD_I64LE
DATASPACE SIMPLE { ( 18 ) / ( 18 ) }
}
DATASET "Values" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 18 ) / ( 18 ) }
}
}
DATASET "Types" {
DATATYPE H5T_STD_U8LE
DATASPACE SIMPLE { ( 7496 ) / ( 7496 ) }
}
}
}
}
I’ll update the issue in vtk.