I’m trying to distribute my vtkUnstructuredGrid objects that are in the order of 1 GB ~ 10 GB by using vtkDistributedDataFilter. It’s interesting that I don’t have any problem with small files by using the same pipeline that are in the order of 20 MB ~ 100 MB but for my larger files when I set SetMinimumGhostLevel the filter never finishes it’s job. I’m using it based on this defined function:
import vtk
from mpi4py import MPI
def vtkUnstructuredGridReader(file_name):
reader = vtk.vtkXMLUnstructuredGridReader()
reader.SetFileName(file_name)
reader.Update()
ug = reader.GetOutput()
return ug
def CreateControllerAndCommunicator(comm):
acomm = vtk.vtkMPI4PyCommunicator.ConvertToVTK(comm)
contr = vtk.vtkMPIController()
contr.SetCommunicator(acomm)
return [contr, acomm]
def DistributeData(ug, contr):
distFilter = vtk.vtkDistributedDataFilter()
distFilter.SetInputData(ug)
distFilter.SetController(contr)
distFilter.UseMinimalMemoryOff()
distFilter.SetMinimumGhostLevel(1)
distFilter.Update()
return [distFilter.GetOutput(), distFilter.GetCuts()]
comm = MPI.COMM_WORLD
contr, acomm = CreateControllerAndCommunicator(comm)
rank = acomm.GetLocalProcessId()
ug = vtk.vtkUnstructuredGrid()
if rank == 0:
ug = vtkUnstructuredGridReader('results_snapshot_1567146.vtu')
distributedUG, distributionCuts = DistributeData(ug, contr)
I’m using OpenMPI 4.0.3 and I built my VTK with parallel functionality enabled from master branch of current repository. Any idea what’s going on here?