Building VTK 9.0 fails when using mpi support

Support
1

When building VTK 9.0 with mpi support, the build fails at linking to generate vtkFiltersParallelGeometryJava with the following errors. Building works without any issue when not compiling with mpi support, which leads me to believe that the problem may be in some env variable not exported before building with mpi, and not directly an issue with the cmake script itself. Note that this used to work fine with VTK 8.2.0.

For mpi support, in the spec file, we redefine the target paths for installation:

%define mpiprefix %{_libdir}/mpi/gcc/openmpi3
%define my_prefix %{mpiprefix}
%define my_bindir %{my_prefix}/bin
%define my_libdir %{my_prefix}/%{_lib}/
%define my_incdir %{my_prefix}/include/
%define my_datadir %{my_prefix}/share/

and use these instead of the standard %{_prefix}=/usr, %{_libdir}, and so on which are used for the non-mpi builds.

[ 1199s] [ 93%] Linking CXX shared library ../../lib64/java/vtk-Linux-x86_64/libvtkFiltersParallelGeometryJava.so
[ 1199s] cd /home/abuild/rpmbuild/BUILD/VTK-9.0.0/build/Wrapping/Java && /usr/bin/cmake -E cmake_link_script CMakeFiles/vtkFiltersParallelGeometryJava.dir/link.txt --verbose
=1
[ 1199s] /usr/lib64/mpi/gcc/openmpi2/bin/mpicxx -fPIC -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-prote
ction -Werror=return-type -flto=auto -ffat-lto-objects -DNDEBUG  -O2 -g -DNDEBUG  -Wl,-lc -flto=auto -ffat-lto-objects -Wl,--as-needed -Wl,--no-undefined -Wl,-z,now  -shared
 -Wl,-soname,libvtkFiltersParallelGeometryJava.so -o ../../lib64/java/vtk-Linux-x86_64/libvtkFiltersParallelGeometryJava.so CMakeFiles/vtkFiltersParallelGeometryJava.dir/CMa
keFiles/vtkFiltersParallelGeometryJava/vtkPConnectivityFilterJava.cxx.o CMakeFiles/vtkFiltersParallelGeometryJava.dir/CMakeFiles/vtkFiltersParallelGeometryJava/vtkPDataSetGh
ostGeneratorJava.cxx.o CMakeFiles/vtkFiltersParallelGeometryJava.dir/CMakeFiles/vtkFiltersParallelGeometryJava/vtkPDistributedDataFilterJava.cxx.o CMakeFiles/vtkFiltersParal
lelGeometryJava.dir/CMakeFiles/vtkFiltersParallelGeometryJava/vtkPStructuredGridConnectivityJava.cxx.o CMakeFiles/vtkFiltersParallelGeometryJava.dir/CMakeFiles/vtkFiltersPar
allelGeometryJava/vtkPStructuredGridGhostDataGeneratorJava.cxx.o CMakeFiles/vtkFiltersParallelGeometryJava.dir/CMakeFiles/vtkFiltersParallelGeometryJava/vtkPUniformGridGhost
DataGeneratorJava.cxx.o CMakeFiles/vtkFiltersParallelGeometryJava.dir/CMakeFiles/vtkFiltersParallelGeometryJava/vtkPUnstructuredGridGhostCellsGeneratorJava.cxx.o CMakeFiles/
vtkFiltersParallelGeometryJava.dir/CMakeFiles/vtkFiltersParallelGeometryJava/vtkFiltersParallelGeometryModuleJava.cxx.o  -Wl,-rpath,"\$ORIGIN/../../:\$ORIGIN:/usr/lib64/jvm/
java/lib:/usr/lib64/jvm/java/lib/server" ../../lib64/libvtkFiltersParallelGeometry.so.9.0.0 ../../lib64/java/vtk-Linux-x86_64/libvtkCommonSystemJava.so ../../lib64/java/vtk-
Linux-x86_64/libvtkFiltersParallelJava.so ../../lib64/java/vtk-Linux-x86_64/libvtkParallelMPIJava.so ../../lib64/libvtkFiltersParallel.so.9.0.0 ../../lib64/libvtkFiltersExtr
action.so.9.0.0 ../../lib64/java/vtk-Linux-x86_64/libvtkFiltersGeometryJava.so ../../lib64/libvtkFiltersHybrid.so.9.0.0 ../../lib64/libvtkFiltersGeometry.so.9.0.0 ../../lib6
4/libvtkFiltersModeling.so.9.0.0 ../../lib64/libvtkFiltersSources.so.9.0.0 ../../lib64/java/vtk-Linux-x86_64/libvtkCommonExecutionModelJava.so ../../lib64/libvtkFiltersTextu
re.so.9.0.0 ../../lib64/libvtkFiltersGeneral.so.9.0.0 ../../lib64/libvtkFiltersCore.so.9.0.0 ../../lib64/libvtkParallelMPI.so.9.0.0 ../../lib64/java/vtk-Linux-x86_64/libvtkC
ommonCoreJava.so ../../lib64/libvtkJava.so.9.0.0 /usr/lib64/jvm/java/lib/libjawt.so /usr/lib64/jvm/java/lib/server/libjvm.so ../../lib64/libvtkParallelCore.so.9.0.0 ../../li
b64/libvtkCommonExecutionModel.so.9.0.0 ../../lib64/libvtkCommonDataModel.so.9.0.0 ../../lib64/libvtkCommonSystem.so.9.0.0 ../../lib64/libvtkCommonTransforms.so.9.0.0 ../../
lib64/libvtkCommonMisc.so.9.0.0 ../../lib64/libvtkCommonMath.so.9.0.0 ../../lib64/libvtkCommonCore.so.9.0.0 ../../lib64/libvtksys.so.9.0.0 -ldl -Wl,-rpath-link,/home/abuild/
rpmbuild/BUILD/VTK-9.0.0/build/lib64/java/vtk-Linux-x86_64:/home/abuild/rpmbuild/BUILD/VTK-9.0.0/build/lib64
[ 1199s] /usr/lib64/gcc/x86_64-suse-linux/9/../../../../x86_64-suse-linux/bin/ld: /tmp/libvtkFiltersParallelGeometryJava.so.ou8EgG.ltrans0.ltrans.o: in function `vtkPConnect
ivityFilter_Typecast':
[ 1199s] /home/abuild/rpmbuild/BUILD/VTK-9.0.0/build/Wrapping/Java/CMakeFiles/vtkFiltersParallelGeometryJava/vtkPConnectivityFilterJava.cxx:18: undefined reference to `vtkCo
nnectivityFilter_Typecast'
[ 1199s] collect2: error: ld returned 1 exit status
[ 1199s] make[2]: *** [Wrapping/Java/CMakeFiles/vtkFiltersParallelGeometryJava.dir/build.make:297: lib64/java/vtk-Linux-x86_64/libvtkFiltersParallelGeometryJava.so] Error 1
[ 1199s] make[2]: Leaving directory '/home/abuild/rpmbuild/BUILD/VTK-9.0.0/build'
[ 1199s] make[1]: *** [CMakeFiles/Makefile2:21809: Wrapping/Java/CMakeFiles/vtkFiltersParallelGeometryJava.dir/all] Error 2
[ 1199s] make[1]: *** Waiting for unfinished jobs....

Full spec file and logs here:

Thanks for any help sorting this out.

2

@dgobbi Are we missing links between our Java modules here?

3

I don’t think that we’re missing links. I did a local test build on my ubuntu system with Java and MPI enabled and didn’t see any missing symbols. The differences as compared to the link line above were as follows:

  • my link line used /usr/bin/c++ and linked to libmpi.so, rather than using mpicxx as above.
  • my link line DID NOT include -Wl,–as-needed, -Wl,–no-undefined, -Wl,-z,now

Exactly the same libraries were linked, so maybe these link flags were what caused the build failure?

@badshah400, I’m not very familiar with MPI, is mpicxx just a wrapper around gcc? Why is it being used for the build instead of gcc, especially since (from what I’ve seen) it’s possible to build with gcc and simply link to libmpi?

Perhaps the only problem is that mpicxx is not properly handled by cmake, but then I don’t understand why the problem didn’t also occur with VTK 8.2.

4

It is an opal wrapper [1] for GCC’s g++ configured with the following definitions so it automatically links against the right version of openmpi:

# There can be multiple blocks of configuration data, chosen by
# compiler flags (using the compiler_args key to chose which block
# should be activated.  This can be useful for multilib builds.  See the
# multilib page at:
#    https://github.com/open-mpi/ompi/wiki/compilerwrapper3264
# for more information.

project=Open MPI
project_short=OMPI
version=2.1.6.0.d9b9e59e5c5a
language=C++
compiler_env=CXX
compiler_flags_env=CXXFLAGS
compiler=g++
preprocessor_flags=
compiler_flags_prefix=
compiler_flags=-pthread
linker_flags=
# Note that per https://svn.open-mpi.org/trac/ompi/ticket/3422, we
# intentionally only link in the MPI libraries (ORTE, OPAL, etc. are
# pulled in implicitly) because we intend MPI applications to only use
# the MPI API.
libs= -lmpi
libs_static= -lmpi -lopen-rte -lopen-pal -lm -lnuma -ldl -lutil  -lrt
dyn_lib_file=libmpi.so
static_lib_file=libmpi.a
required_file=
includedir=${includedir}
libdir=${libdir}

[1] compilerwrapper3264 · open-mpi/ompi Wiki · GitHub

5

Rather than using it as CXX, use it as the MPI_CXX_COMPILER variable (same with mpicc and MPI_C_COMPILER). CMake will extract the information it needs from the wrapper and use it only where we need the MPI information (rather than using it even in vtkCommonCore too).

6

BTW, compiler wrappers are bad in general. I really don’t recommend them for general usage (unless you’re actually in an MPI-saturated stack). The basic problem is that they don’t compose at all. For example, HDF5 has wrappers too. How does one use those and the MPI wrappers at the same time?

7

After further investigation, though, I think the problem is simply a mistake in the dependencies for ParallelGeometry, which can be fixed with the following change:

NAME
   VTK::FiltersParallelGeometry
 LIBRARY_NAME
   vtkFiltersParallelGeometry
 KIT
   VTK::Parallel
 GROUPS
   MPI
 IMPLEMENTS
   VTK::FiltersCore
 DEPENDS
   VTK::CommonCore
   VTK::CommonSystem
   VTK::CommonExecutionModel
+  VTK::FiltersCore
   VTK::FiltersGeometry
   VTK::FiltersParallel
   VTK::ParallelMPI

The -Wl,--no-undefined causes the mistake to become visible during the build.

8 
   VTK::CommonExecutionModel
+ VTK::FiltersCore
   VTK::FiltersGeometry

Big thanks, that did it! I’ll also follow Ben’s advice and use MPI_C*_COMPILER variables instead of redefining CXX and CC.