vtkHDFWriter for temporal PDS Collection

citronella3alain · June 30, 2025, 6:09am

I’m trying to recreate How to write time dependent data in VTKHDF files using the vtkHDFWriter instead of direct h5py wrangling.

I’ve successfully done the following:

write a non-temporal partitioned data set collection using the vtk hdf writer
use the vtk hdf writer to write a sphere polydata that undergoes the same transformations as those used in the example (with one sphere rather than two)

But I’m having trouble writing a temporal partitioned dataset collection with two spheres, i.e., the pipeline given in this example:
Here’s what I currently have appended to the pipeline code, i.e., after warp.Update():

# since vtkhdfwriter requires a data assembly when writing a partitioned dataset collection, create a helper function that creates this data assembly
        def create_data_assembly():
                assembly = vtkDataAssembly()
                root = assembly.GetRootNode()
                assembly.AddNode("Sphere0", root)
                assembly.AddNode("Sphere1", root)
                return assembly

    # Update once to get PDC from pipeline
    warp.Update()
    pdc = warp.GetOutputDataObject(0)
    pdc.SetDataAssembly(create_data_assembly())

    writer = vtkHDFWriter()
    writer.SetInputConnection(warp.GetOutputPort())
    writer.SetFileName(str(path))
    writer.SetWriteAllTimeSteps(True)
    writer.Write()

But I currently get this error:

Temporal polydata initialization failed for PolyData transient.vtkhdf

Thoughts? Thanks!

lgivord · June 30, 2025, 9:13am

Hello @citronella3alain,

Indeed we need an assembly, like that I don’t know what could be your issue. Before writing a temporal pdc, are you able to write a non temporal pdc ?

Additionally, if you can share your data and your script it would be helpful.

FYI @Louis_Gombert

citronella3alain · June 30, 2025, 11:51pm

Hi @lgivord ,
Thanks for your response! Yes, I am able to write a non-temporal partitioned data set collection, as follows:

# this works
def write_vtk_hdf_nontransient(path: Path):
    sphere = vtkSphereSource()
    sphere.SetThetaResolution(20)
    sphere.SetPhiResolution(20)
    sphere.Update()

    surf_polydata = sphere.GetOutput()

    # Convert surface to unstructured grid (for demo)
    tri_filter = vtkTriangleFilter()
    tri_filter.SetInputData(surf_polydata)
    tri_filter.Update()
    volume_ugrid = vtkUnstructuredGrid()
    volume_ugrid.ShallowCopy(tri_filter.GetOutput())

    # Create a surface mesh (PolyData) as-is
    surface = surf_polydata

    # Create a vtkPartitionedDataSetCollection
    pdc = vtkPartitionedDataSetCollection()
    pdc.Initialize()

    # Partition 0 = volume mesh
    pdc.SetNumberOfPartitionedDataSets(2)
    pdc.SetNumberOfPartitions(0, 1)
    pdc.SetNumberOfPartitions(1, 1)

    pdc.SetPartition(0, 0, volume_ugrid)
    pdc.GetMetaData(0).Set(vtkDataObject.FIELD_NAME(), "Volume")

    # Partition 1 = surface mesh
    pdc.SetPartition(1, 0, surface)
    pdc.GetMetaData(1).Set(vtkDataObject.FIELD_NAME(), "Surface")

    assembly = vtkDataAssembly()
    root = assembly.GetRootNode()
    assembly.AddNode("Volume", root)
    assembly.AddNode("Surface", root)

    pdc.SetDataAssembly(assembly)

    writer = vtkHDFWriter()
    writer.SetFileName(str(path))
    # Important: set input as the PDC
    writer.SetInputDataObject(pdc)

    writer.Write()
    return pdc

As for the temporal dataset, I’m currently using the pipeline defined in How to write time dependent data in VTKHDF files because I want to replicate this except using vtkhdfwriter. Here’s the complete code that I currently have (this code does not work properly, this is what I’m trying to fix):

# this does not work properly
from pathlib import Path
def write_transient_pds(path: Path | None):
    sphere0 = vtkSphereSource()
    sphere0.SetPhiResolution(30)
    sphere0.SetThetaResolution(30)
    sphere0.SetRadius(10)

    sphere1 = vtkSphereSource()
    sphere1.SetPhiResolution(30)
    sphere1.SetThetaResolution(30)
    sphere1.SetRadius(10)
    sphere1.SetCenter(15, 15, 15)

    # store the spheres in a single partitioned data set
    groupDataSets = vtkGroupDataSetsFilter()
    groupDataSets.AddInputConnection(sphere0.GetOutputPort())
    groupDataSets.AddInputConnection(sphere1.GetOutputPort())
    groupDataSets.SetOutputTypeToPartitionedDataSetCollection()

    # generate time steps
    timeSteps = vtkGenerateTimeSteps()
    timeSteps.SetInputConnection(groupDataSets.GetOutputPort())
    timeValues = np.linspace(0.0, 2*np.pi, 100, endpoint=False)
    timeSteps.SetTimeStepValues(100, timeValues)

    # generate fields
    addFields = vtkSpatioTemporalHarmonicsAttribute()
    harmonics = np.array([
        [1.0, 1.0, 0.6283, 0.6283, 0.6283, 0.0],
        [3.0, 1.0, 0.6283, 0.0, 0.0, 1.5708],
        [2.0, 2.0, 0.0, 0.6283, 0.0, 3.1416],
        [1.0, 3.0, 0.0, 0.0, 0.6283, 4.7124]
        ])
    for iH in range(harmonics.shape[0]):
        addFields.AddHarmonic(harmonics[iH, 0],
                            harmonics[iH, 1],
                            harmonics[iH, 2],
                            harmonics[iH, 3],
                            harmonics[iH, 4],
                            harmonics[iH, 5])

    addFields.SetInputConnection(timeSteps.GetOutputPort())

    # warp spheres
    warp = vtkWarpScalar()
    warp.SetInputConnection(addFields.GetOutputPort())
    warp.SetInputArrayToProcess(0, 0, 0,
                                vtkDataObject.FIELD_ASSOCIATION_POINTS,
                                'SpatioTemporalHarmonics')
    warp.Update()
    """
    # pdsc = warp.GetOutputDataObject(0)

    # assembly = vtkDataAssembly()
    # root = assembly.GetRootNode()
    # assembly.AddNode("sphere1", root)
    # assembly.AddNode("sphere2", root)

    # pdsc.SetDataAssembly(assembly)
    """

    if path:
        writer = vtkHDFWriter()
        writer.SetFileName(str(path))
        # Important: set input as the PDC
        # writer.SetInputDataObject(pdsc)
        writer.SetInputConnection(warp.GetOutputPort())
        writer.SetWriteAllTimeSteps(True)

        writer.Write()
    # return pdsc
    return warp

One change from the original blog post is that I changed the vtkGroupDataSetsFilter output to pdc rather than just partitioned data set. Another two observations:

If I pass the warp object, the vtkhdf writer dies.
If I extract the output data object from the warp object and persist that, then the persisted object isn’t temporal.

Also, for reference, I have a working example of a temporal polydata using vtkhdfwriter:

# this works
def write_simple_temporal_sphere(output_path: str):
    # 1. Create a basic sphere
    sphere = vtkSphereSource()
    sphere.SetRadius(10)
    sphere.SetThetaResolution(30)
    sphere.SetPhiResolution(30)

    # 2. Generate 2 time steps (e.g., t = 0 and pi)
    time_gen = vtkGenerateTimeSteps()
    time_gen.SetInputConnection(sphere.GetOutputPort())
    time_gen.SetTimeStepValues(2, [0.0, np.pi])

    # 3. Add time-varying field with one harmonic
    harmonics = vtkSpatioTemporalHarmonicsAttribute()
    harmonics.SetInputConnection(time_gen.GetOutputPort())
    harmonics.AddHarmonic(1, 0, 0.5, 0.5, 0.5, 0.0)  # magnitude, frequency, direction, phase

    # 4. Warp the mesh using the harmonic field
    warp = vtkWarpScalar()
    warp.SetInputConnection(harmonics.GetOutputPort())
    warp.SetInputArrayToProcess(0, 0, 0, vtkDataObject.FIELD_ASSOCIATION_POINTS, "SpatioTemporalHarmonics")

    # 5. Write to .vtkhdf with all timesteps
    writer = vtkHDFWriter()
    writer.SetInputConnection(warp.GetOutputPort())
    writer.SetFileName(output_path)
    writer.SetWriteAllTimeSteps(True)
    writer.Write()

So tl;dr: I’m able to write a temporal polydata and I’m able to write a nontemporal partitioned dataset collection, but I don’t know how to write a temporal partitioned dataset collection to vtkhdf. Let me know what additional info I can provide.
Thanks again!

lgivord · July 7, 2025, 7:27am

Hi @citronella3alain,

I modify a bit your script to generate the assembly and it seems to work as expected with vtk 9.5.0 :
test.py (3.4 KB)

let me know if it works on your side too.

here is what I have with ParaView master :

citronella3alain · July 9, 2025, 8:47pm

@lgivord Thanks for taking a look.
While the partitioned data set does write, it isn’t temporal even though “write all timesteps” is enabled.

As a comparison, the temporal polydata I have above does the following:

That said, I took a look at the temporal sphere in Paraview and I realized that while it has timesteps (unlike temporal pdsc), it only recorded the 0th one. So… I’m still confused how writing temporal data works.

I also found out that the vtkhdfreader doesn’t seem able to read the vtkhdf file created here: How to write time dependent data in VTKHDF files. I get this error:

So yeah, I guess I’m generally confused how to use the vtk hdf writer to encode the different points across timesteps in a mesh.

lgivord · July 21, 2025, 6:37am

@Louis_Gombert any idea ?

Louis_Gombert · July 21, 2025, 6:50am

“unable to lock the file” usually means that the file was not closed properly after writing or reading

Louis_Gombert · July 21, 2025, 6:58am

And the dataset is not temporal because we’re breaking the VTK pipeline by adding an assembly manually; writer.SetInputData(pdc) should be writer.SetInputConnection(warp.GetOutputPort()), so the execution pipeline is aware of time steps. Of course, this does not work, because warp does not have an assembly. This is why, for our testing, we created a filter adding assembly to a PDC. Until Group Datasets outputs an assembly, the best option is to create the same filter in Python.