[ARROW-17057] [Python] S3FileSystem has no parameter for retry strategy

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 6.0.1, 7.0.0, 8.0.0
• Fix Version/s: 10.0.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/32365
• Assignee: Duncan
• Reporter: Duncan
• Created: 12/Jul/22 16:02
• Updated: 11/Jan/23 11:48
• Resolved: 10/Aug/22 13:32

Description

The Python wrapper for S3Fs does not accept a retry_strategy parameter, but the underlying C++ implementation supports it.

 

Python wrapper's constructor arguments:
https://github.com/apache/arrow/blob/master/python/pyarrow/_s3fs.pyx#L181
 
C++ base: https://github.com/apache/arrow/blob/master/cpp/src/arrow/filesystem/s3fs.cc#L729
 
The result is that Python users of S3Fs always default to the legacy retry strategy, which is very limited.
 
Suggested fix is to allow the Python wrapper to specify a retry strategy to be passed through to the wrapped C++ implementation.

Issue Links

• links to: GitHub Pull Request #13633