Hi VTK community,
I’m working with AMR data output from the RAMSES astrophysical simulation code. I’m using Python with VTK to convert this data into the vtkOverlappingAMR format for visualization.
Currently, each AMR cell ends up generating a separate .vti file, which results in thousands of tiny .vti files — one for every data point. This makes file I/O and visualization inefficient.
I’m trying to reduce the number of .vti files by:
- Grouping nearby AMR cells into larger uniform grids wherever possible
- Creating smaller grids only for leftover or isolated points
- Still maintaining the level-based AMR hierarchy required by
vtkOverlappingAMR
Has anyone tackled this kind of problem before? Any suggestions on:
- Efficient spatial binning strategies that align with VTK’s AMR requirements
- Possible existing tools or methods in VTK or elsewhere that could help optimize this process
- Best practices for aggregating AMR blocks while preserving correctness
I’ve provided a simplified version of my current Python script that builds the vtkOverlappingAMR structure using Osyris + VTK. If you have any feedback on how I could restructure or optimize the code to reduce the number of .vti files, it would be very helpful.
I’m using:
- VTK version: 9.4.2
- Python version: 3.11.9
- Platform: Linux
Any support, pointers, or example approaches would be greatly appreciated!
import vtk
import osyris
import numpy as np
import pandas as pd
# Load RAMSES AMR data using Osyris
data = osyris.RamsesDataset(1).load()
mesh_data = data["mesh"]
# Initialize arrays to store AMR cell information
x, y, z, length, spacing, density, level = [], [], [], [], [], [], []
corner_coords = []
# Loop through each cell to extract spatial and scalar info
for i in range(len(mesh_data["density"].values)):
px = mesh_data["position"][i].x.values
py = mesh_data["position"][i].y.values
pz = mesh_data["position"][i].z.values
l = float(mesh_data["dx"][i].values)
lev = mesh_data["level"][i].values
d = float(mesh_data["density"][i].values)
space = [l, l, l]
spacing.append(space)
x.append(float(px))
y.append(float(py))
z.append(float(pz))
length.append(l)
density.append(d)
level.append(lev)
# Compute lower corner of the cell
half_l = l / 2
point1 = [float(px)-half_l, float(py)-half_l, float(pz)-half_l]
corner_coords.append(point1)
# Organize cell data into a DataFrame
df = pd.DataFrame({
"level": level,
"corner_coords": corner_coords,
"spacing": spacing
})
# Split DataFrame by AMR level
df_list = []
unique_levels = sorted(df["level"].unique())
for lev in unique_levels:
df_level = df[df["level"] == lev].reset_index(drop=True)
df_list.append(df_level)
# Initialize vtkOverlappingAMR with correct structure
block_values = [len(df_level) for df_level in df_list]
amr = vtk.vtkOverlappingAMR()
amr.Initialize(len(unique_levels), block_values)
amr.SetOrigin([0.0, 0.0, 0.0])
# Set level-wise spacing (assumes uniform spacing per level)
unique_spaces = [df[df["level"] == lev]["spacing"].iloc[0] for lev in unique_levels]
for i, space in enumerate(unique_spaces):
amr.SetSpacing(i, space)
# Create grids for each AMR block
for i, df_level in enumerate(df_list):
for j, row in df_level.iterrows():
grid = vtk.vtkUniformGrid()
grid.SetDimensions(11, 11, 11)
grid.SetOrigin(row["corner_coords"])
grid.SetSpacing(row["spacing"])
# Calculate AMRBox dimensions (indices in global grid)
var = row["corner_coords"]
space = row["spacing"]
i_min, j_min, k_min = int(var[0]/space[0]), int(var[1]/space[1]), int(var[2]/space[2])
dx, dy, dz = 9, 9, 9 # Number of cells per dimension
i_max, j_max, k_max = i_min + dx, j_min + dy, k_min + dz
box = vtk.vtkAMRBox()
box.SetDimensions([i_min, i_max, j_min, j_max, k_min, k_max])
amr.SetAMRBox(i, j, box)
amr.SetDataSet(i, j, grid)
# Write the AMR data to a .vthb file
writer = vtk.vtkXMLUniformGridAMRWriter()
writer.SetFileName("AMR_4.vthb")
writer.SetInputData(amr)
writer.Write()
Thanks.