Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-17064

[Python] Python hangs when use pyarrow.fs.copy_files with "used_threads=True"

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 8.0.0
    • None
    • Python
    • None

    Description

      When try to copy a local path to s3 remote filesystem using `pyarrow.fs.copy_files` and using default parameter `use_threads=True`, the system hangs. If use "use_threads=False` the operation must complete ok (but more slow).

       

      My code is:

      >>> import pyarrow as pa
      >>> s3fs=pa.fs.S3FileSystem(endpoint_override="http://xxxxxx")
      >>> pa.fs.copy_files("tests/data/payments", "bucket/payments", destination_filesystem=s3fs)
      ... (don't return)

      If check remote s3, all files appear, but the function don't return

       

      Platform: Windows

      Attachments

        Activity

          People

            Unassigned Unassigned
            alex_marco Alejandro Marco Ramos
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: