Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-344

Instructions for building with conda



    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.1.0
    • Fix Version/s: 0.2.0
    • Component/s: Python
    • Labels:


      According to this comment, Arrow 0.1.0 for Python can be installed with Conda. arrow-cpp is a dependency of the Python version, and I can install arrow-cpp locally with

      conda install --channel conda-forge parquet-cpp numpy pandas pytest
      cd apache-arrow-0.1.0/cpp
      conda-build conda.recipe --channel conda-forge
      conda install -c conda-forge --use-local arrow-cpp
      cd ../python

      but I can't build and locally install the conda.recipe in the Python directory because conda keeps trying to get the arrow-cpp on conda-forge, rather than the one in the 0.1.0 release. Those versions are incompatible due to a changed API:

      [ 24%] Building CXX object CMakeFiles/pyarrow.dir/src/pyarrow/adapters/builtin.cc.o
      /usr/bin/c++   -Dpyarrow_EXPORTS -isystem /opt/miniconda2/conda-bld/conda.recipe_1476908391204/_b_env_placehold_placehold_/lib/python2.7/site-packages/numpy/core/include -isystem /opt/miniconda2/conda-bld/conda.recipe_1476908391204/_b_env_placehold_placehold_/include/python2.7 -isystem /opt/apache-arrow-0.1.0/python/src -isystem /opt/miniconda2/conda-bld/conda.recipe_1476908391204/_b_env_placehold_placehold_/include  -std=c++11 -Wall -ggdb -O0 -g -fPIC   -fPIC -o CMakeFiles/pyarrow.dir/src/pyarrow/adapters/builtin.cc.o -c /opt/apache-arrow-0.1.0/python/src/pyarrow/adapters/builtin.cc
      /opt/apache-arrow-0.1.0/python/src/pyarrow/adapters/builtin.cc: In function 'pyarrow::Status pyarrow::ConvertPySequence(PyObject*, std::shared_ptr<arrow::Array>*)':
      /opt/apache-arrow-0.1.0/python/src/pyarrow/adapters/builtin.cc:434:26: error: no matching function for call to 'arrow::ArrayBuilder::Finish()'
         *out = builder->Finish();
      /opt/apache-arrow-0.1.0/python/src/pyarrow/adapters/builtin.cc:434:26: note: candidate is:
      In file included from /opt/miniconda2/conda-bld/conda.recipe_1476908391204/_b_env_placehold_placehold_/include/arrow/api.h:24:0,
                       from /opt/apache-arrow-0.1.0/python/src/pyarrow/adapters/builtin.cc:23:
      /opt/miniconda2/conda-bld/conda.recipe_1476908391204/_b_env_placehold_placehold_/include/arrow/builder.h:96:18: note: virtual arrow::Status arrow::ArrayBuilder::Finish(std::shared_ptr<arrow::Array>*)
         virtual Status Finish(std::shared_ptr<Array>* out) = 0;
      /opt/miniconda2/conda-bld/conda.recipe_1476908391204/_b_env_placehold_placehold_/include/arrow/builder.h:96:18: note:   candidate expects 1 argument, 0 provided
      make[2]: *** [CMakeFiles/pyarrow.dir/src/pyarrow/adapters/builtin.cc.o] Error 1
      make[2]: Leaving directory `/opt/apache-arrow-0.1.0/python/build/temp.linux-x86_64-2.7'
      make[1]: *** [CMakeFiles/pyarrow.dir/all] Error 2
      make[1]: Leaving directory `/opt/apache-arrow-0.1.0/python/build/temp.linux-x86_64-2.7'
      make: *** [all] Error 2
      error: command 'make' failed with exit status 2

      If I do conda-build -channel local --channel conda-forge --override-channels, it can't find some of the packages I've installed. If I don't -override-channels, it tries to use arrow-cpp 0.1.post-1 from conda-forge as the dependency and I get the compilation error above.

      Note: my conda list is

      # packages in environment at /opt/miniconda2:
      conda-build               2.0.6                    py27_0
      blas                      1.1                    openblas    conda-forge
      conda                     4.1.12                   py27_0    conda-forge
      conda-env                 2.5.2                    py27_0    conda-forge
      numpy                     1.11.2          py27_blas_openblas_200  [blas_openblas]  conda-forge
      openblas                  0.2.18                        5    conda-forge
      pandas                    0.19.0              np111py27_0    conda-forge
      parquet-cpp               0.1.pre                       3    conda-forge
      pytest                    3.0.3                    py27_0    conda-forge
      thrift-cpp                0.9.3                         3    conda-forge
      enum34                    1.1.6                    py27_0
      filelock                  2.0.6                    py27_0
      jinja2                    2.8                      py27_1
      libgfortran               3.0.0                         1
      arrow-cpp                 0.1                           0    local
      markupsafe                0.23                     py27_2
      mkl                       11.3.3                        0
      openssl                   1.0.2h                        1
      patchelf                  0.9                           0
      pip                       8.1.2                    py27_0
      pkginfo                   1.3.2                    py27_0
      py                        1.4.31                   py27_0
      pycosat                   0.6.1                    py27_1
      pycrypto                  2.6.1                    py27_4
      python                    2.7.12                        1
      python-dateutil           2.5.3                    py27_0
      pytz                      2016.7                   py27_0
      pyyaml                    3.11                     py27_4
      readline                  6.2                           2
      requests                  2.10.0                   py27_0
      ruamel_yaml               0.11.14                  py27_0
      setuptools                23.0.0                   py27_0
      six                       1.10.0                   py27_0
      sqlite                    3.13.0                        0
      tk                        8.5.18                        0
      wheel                     0.29.0                   py27_0
      yaml                      0.1.6                         0
      zlib                      1.2.8                         3

      I'm pretty sure the problem here is something I don't know about conda (I started using it this morning), but I can't figure out how to install this package out-of-the-box.




            • Assignee:
              wesm Wes McKinney
              jpivarski Jim Pivarski
            • Votes:
              0 Vote for this issue
              2 Start watching this issue


              • Created: