This is the “terrible rpath” issue on macOS. The library says “you need to use rpath to use me” (with the @rpath/ library id), but nothing says what path to actually add. Here, you’ll need to do it manually. See how CI does it.
I have discussed it in the CMake Discourse before (and maybe even an issue), but libraries will need an IMPORTED_RPATH property to be set so that CMake can know to add the right -Wl,-rpath, flags to the linker.