Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-8734

[R] improve nightly build installation

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.0.0
    • R

    Description

      I've tried to install / build from source (with from a git checkout and using the built-in `install_arrow()`) and when compiling I'm getting the following error reliably during the auto brew process:

       x System command 'R' failed, exit status: 1, stdout + stderr:
      E> * checking for file ‘/Users/jkeane/Dropbox/arrow/r/DESCRIPTION’ ... OK
      E> * preparing ‘arrow’:
      E> * checking DESCRIPTION meta-information ... OK
      E> * cleaning src
      E> * running ‘cleanup’
      E> * installing the package to build vignettes
      E>       -----------------------------------
      E> * installing *source* package ‘arrow’ ...
      E> ** using staged installation
      E> *** Generating code with data-raw/codegen.R
      E> There were 27 warnings (use warnings() to see them)
      E> *** > 375 functions decorated with [[arrow|s3::export]]
      E> *** > generated file `src/arrowExports.cpp`
      E> *** > generated file `R/arrowExports.R`
      E> *** Downloading apache-arrow
      E> **** Using local manifest for apache-arrow
      E> Thu May  7 13:13:42 CDT 2020: Auto-brewing apache-arrow in /var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow...
      E> ==> Tapping autobrew/core from https://github.com/autobrew/homebrew-core
      E> Tapped 2 commands and 4639 formulae (4,888 files, 12.7MB).
      E> lz4
      E> openssl
      E> thrift
      E> snappy
      E> ==> Downloading https://homebrew.bintray.com/bottles/lz4-1.8.3.mojave.bottle.tar.gz
      E> Already downloaded: /var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/downloads/b4158ef68d619dbf78935df6a42a70b8339a65bc8876cbb4446355ccd40fa5de--lz4-1.8.3.mojave.bottle.tar.gz
      E> ==> Pouring lz4-1.8.3.mojave.bottle.tar.gz
      E> ==> Skipping post_install step for autobrew...
      E> 🍺  /private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/build-apache-arrow/Cellar/lz4/1.8.3: 22 files, 512.7KB
      E> ==> Downloading https://homebrew.bintray.com/bottles/openssl-1.0.2p.mojave.bottle.tar.gz
      E> Already downloaded: /var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/downloads/fbb493745981c8b26c0fab115c76c2a70142bfde9e776c450277e9dfbbba0bb2--openssl-1.0.2p.mojave.bottle.tar.gz
      E> ==> Pouring openssl-1.0.2p.mojave.bottle.tar.gz
      E> ==> Skipping post_install step for autobrew...
      E> ==> Caveats
      E> openssl is keg-only, which means it was not symlinked into /private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/build-apache-arrow,
      E> because Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries.
      E> 
      E> If you need to have openssl first in your PATH run:
      E>   echo 'export PATH="/private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/build-apache-arrow/opt/openssl/bin:$PATH"' >> ~/.zshrc
      E> 
      E> For compilers to find openssl you may need to set:
      E>   export LDFLAGS="-L/private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/build-apache-arrow/opt/openssl/lib"
      E>   export CPPFLAGS="-I/private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/build-apache-arrow/opt/openssl/include"
      E> 
      E> For pkg-config to find openssl you may need to set:
      E>   export PKG_CONFIG_PATH="/private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/build-apache-arrow/opt/openssl/lib/pkgconfig"
      E> 
      E> ==> Summary
      E> 🍺  /private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/build-apache-arrow/Cellar/openssl/1.0.2p: 1,793 files, 12MB
      E> ==> Downloading https://homebrew.bintray.com/bottles/thrift-0.11.0.mojave.bottle.tar.gz
      E> Already downloaded: /var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/downloads/7e05ea11a9f7f924dd7f8f36252ec73a24958b7f214f71e3752a355e75e589bd--thrift-0.11.0.mojave.bottle.tar.gz
      E> ==> Pouring thrift-0.11.0.mojave.bottle.tar.gz
      E> ==> Skipping post_install step for autobrew...
      E> ==> Caveats
      E> To install Ruby binding:
      E>   gem install thrift
      E> ==> Summary
      E> 🍺  /private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/build-apache-arrow/Cellar/thrift/0.11.0: 102 files, 7MB
      E> ==> Downloading https://homebrew.bintray.com/bottles/snappy-1.1.7_1.mojave.bottle.tar.gz
      E> Already downloaded: /var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/downloads/1f09938804055499d1dd951b13b26d80c56eae359aa051284bf4f51d109a9f73--snappy-1.1.7_1.mojave.bottle.tar.gz
      E> ==> Pouring snappy-1.1.7_1.mojave.bottle.tar.gz
      E> ==> Skipping post_install step for autobrew...
      E> 🍺  /private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/build-apache-arrow/Cellar/snappy/1.1.7_1: 18 files, 115.8KB
      E> ==> Downloading https://autobrew.github.io/bottles/apache-arrow-0.17.0.el_capitan.bottle.tar.gz
      E> Already downloaded: /var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/downloads/7dcf2302ba174a5efb32eaa5b8fe0ae874f4a4671f575e126c79a524830054ae--apache-arrow-0.17.0.el_capitan.bottle.tar.gz
      E> ==> Pouring apache-arrow-0.17.0.el_capitan.bottle.tar.gz
      E> ==> Skipping post_install step for autobrew...
      E> 🍺  /private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/build-apache-arrow/Cellar/apache-arrow/0.17.0: 294 files, 49.7MB
      E> ==> Caveats
      E> ==> openssl
      E> openssl is keg-only, which means it was not symlinked into /private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/build-apache-arrow,
      E> because Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries.
      E> 
      E> If you need to have openssl first in your PATH run:
      E>   echo 'export PATH="/private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/build-apache-arrow/opt/openssl/bin:$PATH"' >> ~/.zshrc
      E> 
      E> For compilers to find openssl you may need to set:
      E>   export LDFLAGS="-L/private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/build-apache-arrow/opt/openssl/lib"
      E>   export CPPFLAGS="-I/private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/build-apache-arrow/opt/openssl/include"
      E> 
      E> For pkg-config to find openssl you may need to set:
      E>   export PKG_CONFIG_PATH="/private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/build-apache-arrow/opt/openssl/lib/pkgconfig"
      E> 
      E> ==> thrift
      E> To install Ruby binding:
      E>   gem install thrift
      E> created /var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/lib/libbrewarrow.a
      E> created /var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/lib/libbrewarrow_dataset.a
      E> created /var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/lib/libbrewparquet.a
      E> created /var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/lib/libbrewlz4.a
      E> created /var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/lib/libbrewcrypto.a
      E> created /var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/lib/libbrewssl.a
      E> created /var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/lib/libbrewsnappy.a
      E> created /var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/lib/libbrewthrift.a
      E> created /var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/lib/libbrewthriftz.a
      E> PKG_CFLAGS=-I/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/opt/apache-arrow/include -DARROW_R_WITH_ARROW
      E> PKG_LIBS=-L/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/opt/apache-arrow/lib -L/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/lib -lbrewparquet -lbrewarrow_dataset -lbrewarrow -lbrewthrift -lbrewlz4 -lbrewsnappy
      E> ** libs
      E> clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/opt/apache-arrow/include -DARROW_R_WITH_ARROW -I'/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include' -I/usr/local/include   -fPIC  -Wall -g -O2  -c array.cpp -o array.o
      E> clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/opt/apache-arrow/include -DARROW_R_WITH_ARROW -I'/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include' -I/usr/local/include   -fPIC  -Wall -g -O2  -c array_from_vector.cpp -o array_from_vector.o
      E> clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/opt/apache-arrow/include -DARROW_R_WITH_ARROW -I'/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include' -I/usr/local/include   -fPIC  -Wall -g -O2  -c array_to_vector.cpp -o array_to_vector.o
      E> clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/opt/apache-arrow/include -DARROW_R_WITH_ARROW -I'/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include' -I/usr/local/include   -fPIC  -Wall -g -O2  -c arraydata.cpp -o arraydata.o
      E> clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T//build-apache-arrow/opt/apache-arrow/include -DARROW_R_WITH_ARROW -I'/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include' -I/usr/local/include   -fPIC  -Wall -g -O2  -c arrowExports.cpp -o arrowExports.o
      E> arrowExports.cpp:1650:21: error: no member named 'CsvFileFormat' in namespace 'arrow::dataset'
      E> std::shared_ptr<ds::CsvFileFormat> dataset___CsvFileFormat__Make(const std::shared_ptr<arrow::csv::ParseOptions>& parse_options);
      E>                 ~~~~^
      E> 1 error generated.
      E> make: *** [arrowExports.o] Error 1
      E> ERROR: compilation failed for package ‘arrow’
      E> * removing ‘/private/var/folders/45/n5gfjjtn05j877spnpbnhqqw0000gn/T/RtmpNwJEKk/Rinstd9644c71dcbf/arrow’
      E>       -----------------------------------
      E> ERROR: package installation failed 
      

      I'm sure I'm missing some setup step somewhere or I'm missing a necessary step to install from source. I've tried this both under R 3.6(.1) and 4.0 to see if upgrading helped and I get the same error

      Attachments

        Issue Links

          Activity

            People

              npr Neal Richardson
              jonkeane Jonathan Keane
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 50m
                  50m