Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-34979

Failed to install pyspark[sql] (due to pyarrow error) on aarch64

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 3.2.0
    • Fix Version/s: 3.2.0
    • Component/s: Documentation, PySpark
    • Labels:
      None

      Description

      $ pip install pyspark[sql]

      Collecting pyarrow>=1.0.0
      Using cached pyarrow-3.0.0.tar.gz (682 kB)
      Installing build dependencies ... done
      Getting requirements to build wheel ... done
      Preparing wheel metadata ... done
      // ... ...
      Building wheels for collected packages: pyarrow
      Building wheel for pyarrow (PEP 517) ... error
      ERROR: Command errored out with exit status 1:
      command: /root/venv/bin/python3.8 /tmp/tmpv35m1o0g build_wheel /tmp/tmpq0n5juib
      cwd: /tmp/pip-install-sh0myu71/pyarrow
      Complete output (183 lines):

      – Running cmake for pyarrow
      cmake -DPYTHON_EXECUTABLE=/root/venv/bin/python3.8 -DPython3_EXECUTABLE=/root/venv/bin/python3.8 -DPYARROW_BUILD_CUDA=off -DPYARROW_BUILD_FLIGHT=off -DPYARROW_BUILD_GANDIVA=off -DPYARROW_BUILD_DATASET=off -DPYARROW_BUILD_ORC=off -DPYARROW_BUILD_PARQUET=off -DPYARROW_BUILD_PLASMA=off -DPYARROW_BUILD_S3=off -DPYARROW_BUILD_HDFS=off -DPYARROW_USE_TENSORFLOW=off -DPYARROW_BUNDLE_ARROW_CPP=off -DPYARROW_BUNDLE_BOOST=off -DPYARROW_GENERATE_COVERAGE=off -DPYARROW_BOOST_USE_SHARED=on -DPYARROW_PARQUET_USE_SHARED=on -DCMAKE_BUILD_TYPE=release /tmp/pip-install-sh0myu71/pyarrow
      error: command 'cmake' failed with exit status 1
      ----------------------------------------
      ERROR: Failed building wheel for pyarrow
      Failed to build pyarrow
      ERROR: Could not build wheels for pyarrow which use PEP 517 and cannot be installed directly

       

      The pip installation would be failed, due to the dependency pyarrow install failed.

       

       

        Attachments

          Activity

            People

            • Assignee:
              yikunkero Yikun Jiang
              Reporter:
              yikunkero Yikun Jiang

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment