[ARROW-6766] [Python] libarrow_python..dylib does not exist

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Cannot Reproduce
• Affects Version/s: 0.14.0, 0.15.0
• Component/s: Python
• External issue URL: https://github.com/apache/arrow/issues/23105
• Assignee: Unassigned
• Reporter: Tarek Allam
• Created: 02/Oct/19 14:16
• Updated: 11/Jan/23 07:49
• Resolved: 26/Feb/20 18:56

Description

After following the instructions found on the developer guides for Python, I was
able to build fine by using:

# Assuming immediately prior one has run:
# $ git clone [e-mail]:apache/arrow.git

1. $ conda create -y -n pyarrow-dev -c conda-forge 
   #   --file arrow/ci/conda_env_unix.yml 
   #   --file arrow/ci/conda_env_cpp.yml 
   #   --file arrow/ci/conda_env_python.yml 
   #    compilers 
   #  python=3.7
   # $ conda activate pyarrow-dev
   # $ brew update && brew bundle --file=arrow/cpp/Brewfileexport ARROW_HOME=$(pwd)/arrow/dist
   export LD_LIBRARY_PATH=$(pwd)/arrow/dist/lib:$LD_LIBRARY_PATHexport CC=`which clang`
   export CXX=`which clang++`{{mkdir arrow/cpp/build }}
       pushd arrow/cpp/build \
       cmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME \
       -DCMAKE_INSTALL_LIBDIR=lib \
       -DARROW_FLIGHT=OFF \
       -DARROW_GANDIVA=OFF \
       -DARROW_ORC=ON \
       -DARROW_PARQUET=ON \
       -DARROW_PYTHON=ON \
       -DARROW_PLASMA=ON \
       -DARROW_BUILD_TESTS=ON \
      ..
   make -j4
   make install
   popd

But when I run:

pushd arrow/python
export PYARROW_WITH_FLIGHT=0
export PYARROW_WITH_GANDIVA=0
export PYARROW_WITH_ORC=1
export PYARROW_WITH_PARQUET=1
python setup.py build_ext --inplace
popd

I get the following errors:

-- Build output directory: /Users/tallamjr/Github/arrow/python/build/temp.macosx-10.9-x86_64-3.7/release
-- Found the Arrow core library: /usr/local/anaconda3/envs/pyarrow-dev/lib/libarrow.dylib
-- Found the Arrow Python library: /usr/local/anaconda3/envs/pyarrow-dev/lib/libarrow_python.dylib
CMake Error: File /usr/local/anaconda3/envs/pyarrow-dev/lib/libarrow..dylib does not exist....CMake Error: File /usr/local/anaconda3/envs/pyarrow-dev/lib/libarrow..dylib does not exist.
CMake Error at CMakeLists.txt:230 (configure_file):
{{ configure_file Problem configuring file}}
Call Stack (most recent call first):
{{ CMakeLists.txt:315 (bundle_arrow_lib)}}
CMake Error: File /usr/local/anaconda3/envs/pyarrow-dev/lib/libarrow_python..dylib does not exist.
CMake Error at CMakeLists.txt:226 (configure_file):
{{ configure_file Problem configuring file}}
Call Stack (most recent call first):
{{ CMakeLists.txt:320 (bundle_arrow_lib)}}
CMake Error: File /usr/local/anaconda3/envs/pyarrow-dev/lib/libarrow_python..dylib does not exist.
CMake Error at CMakeLists.txt:230 (configure_file):
{{ configure_file Problem configuring file}}
Call Stack (most recent call first):
{{ CMakeLists.txt:320 (bundle_arrow_lib)}}

 

What is quite strange is that the libraries seem to indeed be there but they
have an addition component such as `libarrow.15.dylib` .e.g:

$ ls -l libarrow_python.15.dylib && echo $PWD
lrwxr-xr-x 1 tallamjr staff 28 Oct 2 14:02 libarrow_python.15.dylib ->
libarrow_python.15.0.0.dylib
/Users/tallamjr/github/arrow/dist/lib

I guess I am not exactly sure what the issue here is but it appears to be that
the version is not captured as a variable that is used by CMAKE? I have run the
same setup on `master` (`7d18c1c`) and on `apache-arrow-0.14.0` (`a591d76`)
which both seem to produce same errors.

Apologies if this is not quite the format for JIRA issues here or perhaps if
it's not the correct platform for this, I'm very new to the project and
contributing to apache in general. Thanks