Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
Description
We currently vendor/bundle all compression libraries and have them disabled in the default build. This is reliable, but it would be nice to use system libraries if they're present.
It's not as simple as setting ARROW_DEPENDENCY_SOURCE=AUTO because we have to know if we're using them in order to set the right `-lwhatever` flags in the R package build (see ARROW-9303 for what happens if you don't). ARROW-6312 should now track everything we need in the *.pc files, so if we can get that information in our arrow C++ build, we should be able to use AUTO.
Steps/tasks:
- Make ARROW_DEPENDENCY_SOURCE configurable by env var in build_arrow_static.sh but don't change the default (we want this to be opt-in for this release, can change the default for the next release)
- If ARROW_DEPENDENCY_SOURCE=AUTO, we'll need to get the -l flags from the build. The right way is with pkg-config (as Kou showed), so if we want to use it, we'll have to check that pkg-config is installed before allowing AUTO. Then after doing the C++ build, call pkg-config on the install directory to set the PKG_LIBS. This probably needs to be done from r/configure because that's where that env var is set. Alternatively, we could try to grep/sed it from the *.pc files.
- Make sure we have CI that tests both the AUTO and BUNDLED paths.
- Possibly add the change Kou suggested at https://github.com/apache/arrow/pull/10404#issuecomment-871034191 to handle the case where libs were built outside the R package installation (though we should doublecheck what --static does, we don't want this to only work for a static Arrow C++ build, this should also work for shared libraries here)
Attachments
Issue Links
- links to
