Getting rid of modules FiltersParallelDIY2 and FiltersParallel?

The current module architecture around filters using DIY necessitates placing these filters in FiltersParallelDIY2 or in a module for which FiltersParallelDIY2 doesn’t depend on, to avoid cyclic dependencies. Some filters that require DIY may however have other dependencies (such as FiltersTexture). There can be dependencies within FiltersTexture that rely on FiltersParallelDIY2 (for example, vtkGenerateGlobalIds, a general filter that uses DIY). When encountering this kind of interdependency, a filter intended for FiltersTexture but using vtkGenerateGlobalIds must be placed in FiltersParallelDIY2 even if it seems illogical. This module arrangement segments filters based on their parallel backend requirements, particularly in a multi-process context.

I believe categorizing filters based on their “parallelness” is irrelevant. They should be organized based on the type of operation they perform more fundamentally. The current module structure likely stems from an outdated division between regular filters and their parallel subclasses (the “P” filters). This distinction is obsolete because filters can now include a vtkMultiProcessController without requiring MPI to be enabled in the build, functioning instead with a vtkDummyController. Several filters already operate this way (vtkGhostCellsGenerator, vtkGenerateGlobalIds, etc.), needing only access to the vtkMultiProcessController interface and potentially DIY, which performs well without MPI.

Maintaining the current infrastructure will lead to more arbitrary filters, without obvious parallel components, ending up in FiltersParallel or FiltersParallelDIY2. This situation already exists to some extent, with diverse filter types in these modules that might fit better elsewhere. The cyclic dependencies force these filters into these modules by default.

I propose establishing a base infrastructure that supports multi-process communication without relying on a module containing parallel-specific filters. Essentially, we should consider making DIY a more core component, allowing any filter to access it without creating cycles, and have ParallelCore also “more core” (which includes vtkMultiProcessController). We should then reassign the filters currently in FiltersParallel and FiltersParallelDIY2 to more appropriate modules.

I seek feedback on this concern. Have I overlooked any factors that would make this change problematic? Is there a better approach?

As a compatibility layer, we should keep the associated targets (VTK::FiltersParallel) and Python modules (vtkmodules.vtkFiltersParallel) existing, but in a deprecated state while providing the classes that have relocated (Python can be exact with re-exports, but CMake is going to end up exposing the entire module that provides a moved class). This would involve some module work to tell wrappers what to do and to make the targets depend on the moved-to modules.

I agree that the situation with that module is problematic and any steps taken to resolve the issue is good.

Retrocompatibility is paramount though.