[HADOOP-18703] Backport S3A prefetching stream to branch-3.3

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.9
• Fix Version/s: 3.3.6
• Component/s: fs/s3
• Target Version/s: 3.3.9
• Assignee: Steve Loughran Assign to me
• Reporter: Steve Loughran
• Created: 14/Apr/23 16:12
• Updated: 08/Jun/23 20:25
• Resolved: 28/Apr/23 11:46

Description

Jira to cover work for backporting the prefetching into branch-3.3

I'm thinking of the best way to do do this. Proposed

1. new feature branch
2. cherrypick from trunk all current changes for it
3. squash into one single pr, the backport change
4. merge that into branch-3.3
5. then for all the outstanding issues add them separately to trunk and branch-3.3

Issue Links

• depends upon: HADOOP-18186 s3a prefetching to use SemaphoredDelegatingExecutor for submitting work
• depends upon: HADOOP-18380 fs.s3a.prefetch.block.size to be read through longBytesOption
• depends upon: HADOOP-18455 s3a prefetching Executor should be closed
• depends upon: HADOOP-18466 Limit the findbugs suppression IS2_INCONSISTENT_SYNC to S3AFileSystem field
• links to: branch feature-HADOOP-18028-s3a-prefetch-branch-3.3
• links to: https://github.com/apache/hadoop/pull/5559