Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-8983

[Python] Downloading sources of pyarrow and its requirements from pypi takes several minutes starting from 0.16.0

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Won't Fix
    • 0.16.0, 0.17.0, 0.17.1
    • None
    • None
    • None

    Description

      It appears that

      python -m pip download --dest /tmp pyarrow==0.17.1 --no-binary :all:

      takes several minutes to execute.

      There seems to be an increase in runtime starting from 0.16.0: on Python 2

      python -m pip download --dest /tmp pyarrow==0.15.1 --no-binary :all:

      appears to be somewhat faster, but the same command is still slow on Py3.

      The command is stuck for a while with "Installing build dependencies ... ", and increased CPU usage.

      The intent of this command is to download source tarball for a package and its dependencies.

      Some investigation was started on the mailing list: https://lists.apache.org/thread.html/r9baa48a9d1517834c285f0f238f29fcf54405cb7cf1e681314239d7f%40%3Cdev.arrow.apache.org%3E

      Attachments

        Activity

          People

            Unassigned Unassigned
            tvalentyn Valentyn Tymofieiev
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: