Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-11460

[R] Use system libraries if present on Linux

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 6.0.0
    • R

    Description

      We currently vendor/bundle all compression libraries and have them disabled in the default build. This is reliable, but it would be nice to use system libraries if they're present.

      It's not as simple as setting ARROW_DEPENDENCY_SOURCE=AUTO because we have to know if we're using them in order to set the right `-lwhatever` flags in the R package build (see ARROW-9303 for what happens if you don't). ARROW-6312 should now track everything we need in the *.pc files, so if we can get that information in our arrow C++ build, we should be able to use AUTO.

      Steps/tasks:

      • Make ARROW_DEPENDENCY_SOURCE configurable by env var in build_arrow_static.sh but don't change the default (we want this to be opt-in for this release, can change the default for the next release)
      • If ARROW_DEPENDENCY_SOURCE=AUTO, we'll need to get the -l flags from the build. The right way is with pkg-config (as Kou showed), so if we want to use it, we'll have to check that pkg-config is installed before allowing AUTO. Then after doing the C++ build, call pkg-config on the install directory to set the PKG_LIBS. This probably needs to be done from r/configure because that's where that env var is set. Alternatively, we could try to grep/sed it from the *.pc files.
      • Make sure we have CI that tests both the AUTO and BUNDLED paths.
      • Possibly add the change Kou suggested at https://github.com/apache/arrow/pull/10404#issuecomment-871034191 to handle the case where libs were built outside the R package installation (though we should doublecheck what --static does, we don't want this to only work for a static Arrow C++ build, this should also work for shared libraries here)

      Attachments

        Issue Links

          Activity

            People

              thisisnic Nicola Crane
              npr Neal Richardson
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 24h 40m
                  24h 40m