Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-15541

[Python] Cannot build pyarrow with older version of cython

    XMLWordPrintableJSON

Details

    Description

      Hi There,

      I installed arrow/5.0.0 as the following.

      """

      git clone https://github.com/apache/arrow.git

      cd arrow

      git checkout apache-arrow-5.0.0

      export ARROW_HOME=/g/data/z00/yxs900/.local/environments/arrow

      mkdir cpp/build

      cd cpp/build

      cmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME \

            -DCMAKE_INSTALL_LIBDIR=lib64 \

            -DCMAKE_BUILD_TYPE=release \

            -DPython3_EXECUTABLE=/apps/python3/3.7.4/bin/python3 \

            -DARROW_PARQUET=ON \

            -DARROW_PYTHON=ON \

            -DARROW_CUDA=ON \

            -DARROW_DATASET=ON \

            -DARROW_BUILD_TESTS=OFF \

            ..

       

      make -j1 VERBOSE=1

      make install

      """

      The build is successful and installed properly. But when I try to install the python wrapper as the following

      """

      cd ../../python

      export PYARROW_WITH_CUDA=1

      export PYARROW_WITH_DATASET=1

      export PYARROW_WITH_PARQUET=1

      export LD_LIBRARY_PATH=${ARROW_HOME}/lib64:${LD_LIBRARY_PATH}

      python3 setup.py build_ext --inplace

      """

      It stopped at the error message

      """

      Error compiling Cython file:

      ------------------------------------------------------------

      ...

                                                                Writable):

              pass

       

          cdef cppclass CInputStream" arrow::io::InputStream"(FileInterface,

                                                              Readable):

              CResult[shared_ptr[const CKeyValueMetadata]] ReadMetadata()

                                      ^

      ------------------------------------------------------------

       

      pyarrow/includes/libarrow.pxd:1199:33: Expected ']', found 'CKeyValueMetadata'

      """

      Did you see this error before? I am using cmake/3.21.4, cuda/11.0.3, mccl/2.8.4, and python 3.7.4. The cython version is 0.29.14. 

      Let me know what else do you need for the diagnosis.

      Cheers,
      Yue

      Attachments

        Issue Links

          Activity

            People

              jorisvandenbossche Joris Van den Bossche
              sue124 Yue Sun
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 40m
                  2h 40m