Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-9261

[Python][Packaging] S3FileSystem curl errors in manylinux wheels

    XMLWordPrintableJSON

Details

    Description

      https://issues.apache.org/jira/browse/ARROW-9109 introduced S3 support in manylinux wheels. However, when trying to use S3FileSystem it fails with

       

      Traceback (most recent call last):
       File "<string>", line 1, in <module>
       File "pyarrow/_fs.pyx", line 597, in pyarrow._fs.FileSystem.open_input_stream
       File "pyarrow/error.pxi", line 122, in pyarrow.lib.pyarrow_internal_check_status
       File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
      OSError: When reading information for key 'xxxxx' in bucket 'xxxxx': AWS Error [code 99]: curlCode: 77, Problem with the SSL CA cert (path? access rights?) with address

      It seems like it can't find the SSL CA cert directory that is installed in the runtime machine (tested on Ubuntu 16.04 and Ubuntu 18.04). It always searches in /etc/pki/tls/certs/ca-bundle.crt probably because the wheels are built on centos, whereas in Ubuntu the path is /etc/ssl/certs/ca-certificates.crt and is different on other distributions.

      Reproduce with:

      virtualenv -p python3.8 arrowenv
      source arrowenv/bin/activate
      pip install --extra-index-url https://repo.fury.io/arrow-nightlies/ --pre pyarrow
      python -c "from pyarrow.fs import S3FileSystem; fs = S3FileSystem(); fs.open_input_stream('mybucket/myfile')"

       

       

      Attachments

        Issue Links

          Activity

            People

              apitrou Antoine Pitrou
              roee88 Roee Shlomo
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 10m
                  2h 10m