A connectivity filter ( like vtkImageConnectivityFilter) is close to what you describe. It will assign distinct labels to connected regions of the image. You can then pick the ones you want. Make sure to also check vtkImageConnectivityFilter::SetExtractionModeToLargestRegion() as well as earlier posts on discourse . I haven’t checked with vtk.js but I expect this filters to be available.