[HADOOP-15963] Add ABFS support for Async Scatter/Gather IO

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.3.0
• Component/s: fs/azure
• Labels: abfsactive
• Assignee: Anmol Asrani
• Reporter: Steve Loughran
• Created: 03/Dec/18 12:30
• Updated: 27/Feb/24 11:09

Description

HADOOP-11867 is proposing adding a new scatter/gather IO API.

For an object store to take advantage of it, it should be doing things like

• coalescing reads even with a gap between them
• choosing an optimal ordering of requests
• submitting reads into the executor pool/using any async API provided by the FS.
• detecting overlapping reads (and then what?)
• switching to HTTP 2 where supported

Do this for ABFS

Issue Links

• depends upon: HADOOP-11867 Add a high-performance vectored read API.
• depends upon: HADOOP-15229 Add FileSystem builder-based openFile() API to match createFile(); S3A to implement S3 Select through this API.