[HADOOP-15964] Add S3A support for Async Scatter/Gather IO

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.5
• Component/s: fs/s3
• Assignee: Mukund Thakur
• Reporter: Steve Loughran
• Created: 03/Dec/18 12:32
• Updated: 04/Aug/22 12:39
• Resolved: 04/Aug/22 12:39

Description

HADOOP-11867 is proposing adding a new scatter/gather IO API.

For an object store to take advantage of it, it should be doing things like

• coalescing reads even with a gap between them
• choosing an optimal ordering of requests
• submitting reads into the executor pool/using any async API provided by the FS.
• detecting overlapping reads (and then what?)
• switching to HTTP 2 where supported

Do this for S3A

Issue Links

• depends upon: HADOOP-11867 Add a high-performance vectored read API.
• depends upon: HADOOP-15229 Add FileSystem builder-based openFile() API to match createFile(); S3A to implement S3 Select through this API.