[ARROW-6357] [C++] S3: allow for background writes

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.15.0
• Component/s: C++
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/22732
• Assignee: Antoine Pitrou
• Reporter: Antoine Pitrou
• Created: 26/Aug/19 12:53
• Updated: 11/Jan/23 07:46
• Resolved: 12/Sep/19 15:59

Description

S3 writes can be synchronous: when a part is uploaded, Write() will only return once the server has received the contents and responded "Ok".

We may want to add a S3 option to issue writes in the background (e.g. using the S3 client's asynchronous APIs).

Another possibility is to provide a general OutputStream implementation that backgrounds writes (this should be independent of BufferedOutputStream, though, because S3FileSystem already has its own buffering layer).

Issue Links

• links to: GitHub Pull Request #5344