You’ll need to write a projectA-config.cmake file so that projectB can do find_package(projectA). You can see the CMake/vtk-config.cmake.in file in VTK itself for how to set it up.
You can use LibXM2 just like any other external project, though you’ll need to likely export its find module to your CMake package. VTK does this in the VTK::FiltersOpenTURNS module.
You can see how it is done in ParaView, SMTK (which can use ParaView and also provides its own modules). Note that ParaView is a bit weird because it also provides VTK itself, but SMTK should have parts which match what you want to do.