Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-7908

[R] Can't install package without setting LIBARROW_DOWNLOAD=true

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.16.0
    • 0.17.0
    • R
    • None
    • Operating System: Red Hat Enterprise Linux Server 7.6 (Maipo)
      CPE OS Name: cpe:/o:redhat:enterprise_linux:7.6:GA:server
      Kernel: Linux 3.10.0-957.35.2.el7.x86_64
      Architecture: x86-64

    Description

      Hi,

      Installing arrow in R does not work intuitively on our server.

      install.packages("arrow")`
      

      results in an error:

      Installing package into '/home/<user>/R/x86_64-redhat-linux-gnu-library/3.6'
      (as 'lib' is unspecified)
      
      trying URL 'https://cloud.r-project.org/src/contrib/arrow_0.16.0.2.tar.gz'
      Content type 'application/x-gzip' length 216119 bytes (211 KB)
      ==================================================
      downloaded 211 KB
      
      * installing *source* package 'arrow' ...
      ** package 'arrow' successfully unpacked and MD5 sums checked
      ** using staged installation
      PKG_CFLAGS=-I/tmp/Rtmp3v1BDf/R.INSTALL4a5d5d9f8bc8/arrow/libarrow/arrow-0.16.0.2/include  -DARROW_R_WITH_ARROW
      PKG_LIBS=-L/tmp/Rtmp3v1BDf/R.INSTALL4a5d5d9f8bc8/arrow/libarrow/arrow-0.16.0.2/lib -larrow_dataset -lparquet -larrow -lthrift -lsnappy -lz -lzstd -llz4 -lbrotlidec-static -lbrotlienc-static -lbrotlicommon-static -lboost_filesystem -lboost_regex -lboost_system -ljemalloc_pic
      ** libs
      g++ -m64 -std=gnu++11 -I"/usr/include/R" -DNDEBUG -I/tmp/Rtmp3v1BDf/R.INSTALL4a5d5d9f8bc8/arrow/libarrow/arrow-0.16.0.2/include  -DARROW_R_WITH_ARROW -I"/usr/lib64/R/library/Rcpp/include" -I/usr/local/include  -fpic  -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic  -c array.cpp -o array.o
      In file included from array.cpp:18:0:
      ./arrow_types.h:201:31: fatal error: arrow/dataset/api.h: No such file or directory
      

      It appears that the C++ code is not built. With arrow 0.16.0.1 things do work out, because it tries to build the C++ code from source. With arrow 0.16.0.2 such is no longer the case. I could finish the installation by setting the environment variable LIBARROW_DOWNLOAD to 'true':

      export LIBARROW_DOWNLOAD=true
      

      That, apparently, triggers the build from source. I would have expected that I would not need to set this variable explicitly.

      I found that between versions, the default value of LIBARROW_DOWNLOAD has changed:

      - download_ok <- locally_installing && !env_is("LIBARROW_DOWNLOAD", "false")
      + download_ok <- env_is("LIBARROW_DOWNLOAD", "true")
      

      In our environment, that variable was not set, resulting (accidentally?) in download_ok being false and therefore the libraries not being installed and finally the resulting error above.

       

      I can't quite figure out the logic behind all this, but it would be nice if we'd be able to install the package without first having to set LIBARROW_DOWNLOAD.

       

      Thank you for looking into this!

      Attachments

        Activity

          People

            npr Neal Richardson
            taeke Taeke
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment