Excessive RAM Usage and System Crash When Opening Large `.vtkhdf` AMR Dataset in ParaView

Support
1

Hello,

I’m working with astrophysical simulation data and using Python to generate .vtkhdf files based on vtkOverlappingAMR. The code I’m using was created by adapting a sample provided for converting Overlapping AMR data to the .vtkhdf format. The output files load correctly in ParaView 5.13 when the dataset is small. However, when I apply it to moderately larger data, opening the resulting file causes extremely high memory usage on my system, eventually leading to a full system freeze and a Blue Screen of Death (BSOD).

The .vtkhdf file includes scalar fields such as density, pressure and velocity. I’m trying to understand whether this kind of memory issue is expected when working with larger AMR datasets in this format, or if there are known practices to optimize file generation or control loading behavior in ParaView.

Questions:

  1. Are there known memory limitations or considerations when working with large .vtkhdf files containing overlapping AMR data?
  2. Does ParaView load all AMR blocks and fields into memory at once when reading .vtkhdf, or is there a way to control this behavior?
  3. Are there recommended practices for generating or visualizing large .vtkhdf files to avoid such system-level crashes?

Thanks in advance for your guidance!

2

Hello @Hemangi_Varkal,

  1. Are there known memory limitations or considerations when working with large .vtkhdf files containing overlapping AMR data?

Not AFAIK

Does ParaView load all AMR blocks and fields into memory at once when reading .vtkhdf, or is there a way to control this behavior?

Yes, in ParaView if the property Maximum Levels To Read By Default For AMR is set to 0, all block will be read, you can test to read only the first level by setting it to 1.

Are there recommended practices for generating or visualizing large .vtkhdf files to avoid such system-level crashes?

When user see a very high memory consumption which is not expected, the main culprit is the chunk size. See VTK: vtkHDFWriter Class Reference

This is hard to give a good default chunk size value as it depends on the size of the data itself, we have an issue here to improve that at some point : https://gitlab.kitware.com/paraview/paraview/-/issues/22832

3

Thanks for your suggestion. I was able to see the structures after setting the AMR values manually.