Details
-
Bug
-
Status: Closed
-
Minor
-
Resolution: Not A Problem
-
None
-
None
-
None
-
None
Description
Opening a ticket for: https://github.com/apache/arrow/issues/2281 after running into the same problems with RedHat 7.4.
https://arrow.apache.org/docs/python/development.html#development
Additional steps taken:
Added double-conversion, glog and hypothesis:
conda create -y -q -n pyarrow-dev \
python=3.6 numpy six setuptools cython pandas pytest double-conversion \
cmake flatbuffers rapidjson boost-cpp thrift-cpp snappy zlib glog hypothesis\
gflags brotli jemalloc lz4-c zstd -c conda-forge
Added export LD_LIBRARY_PATH to conda lib64 before running py.test pyarrow:
export LD_LIBRARY_PATH=/home/my_login/anaconda3/envs/pyarrow-dev/lib64 py.test pyarrow
Added extra symlinks with a period at the end to fix string concatenation issues. Running setup.py for the first time didn't need this, but running setup.py a second time would error out with:
CMake Error: File /home/my_login/anaconda3/envs/pyarrow-dev/lib64/libarrow.so. does not exist.
There is an extra period at the end of the *.so files so I had to make symlinks with extra periods.
ln -s libparquet.so.12.0.0 libparquet.so. ln -s libplasma.so.12.0.0 libplasma.so. ln -s libarrow.so.12.0.0 libarrow.so. ln -s libarrow_python.so.12.0.0 libarrow_python.so.
Creating a wheel file using --with-plasma gives the following error:
error: [Errno 2] No such file or directory: 'release/plasma_store_server'
Had to create the wheel file without plasma, but it isn't packaged correctly. The hacked symlinked shared libs are included instead of libarrow.so.12
copying build/lib.linux-x86_64-3.6/pyarrow/libarrow.so. -> build/bdist.linux-x86_64/wheel/pyarrow copying build/lib.linux-x86_64-3.6/pyarrow/libarrow.so -> build/bdist.linux-x86_64/wheel/pyarrow copying build/lib.linux-x86_64-3.6/pyarrow/libarrow_python.so. -> build/bdist.linux-x86_64/wheel/pyarrow copying build/lib.linux-x86_64-3.6/pyarrow/libarrow_python.so -> build/bdist.linux-x86_64/wheel/pyarrow copying build/lib.linux-x86_64-3.6/pyarrow/libplasma.so. -> build/bdist.linux-x86_64/wheel/pyarrow copying build/lib.linux-x86_64-3.6/pyarrow/libplasma.so -> build/bdist.linux-x86_64/wheel/pyarrow