[HADOOP-16829] Über-jira: S3A Hadoop 3.3.1 features

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.1
• Component/s: fs/s3
• Target Version/s: 3.3.1
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 27/Jan/20 20:09
• Updated: 11/Nov/21 09:57
• Resolved: 13/May/21 04:25

Description

Über-jira: S3A features/fixes for Hadoop 3.4

As usual, this will clutter up with everything which hasn't gone in: don't interpret presence on this list as a commitment to implement.

And for anyone wanting to add patches

MUST

1. reviews via github PRs
2. no declaration of AWS S3 endpoint (or other S3 impl) -no review

SHOULD

1. have a setup for testing SSE-KMS, DDB/S3Guard
2. including an assumed role we can use for AssumedRole Delegation Tokens

If you are going near those bits of code, they uprate from SHOULD to MUST.

Issue Links

• depends upon: MAPREDUCE-7182 MapReduce input format/record readers to support S3 select queries
• depends upon: HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden
• depends upon: MAPREDUCE-5907 Improve getSplits() performance for fs implementations that can utilize performance gains from recursive listing
• is blocked by: HADOOP-16111 FileSystem.closeAllForUGI closes() filesystem in Sync block
• is blocked by: HADOOP-17519 Decommission S3Guard
• is superceded by: HADOOP-17566 Über-jira: S3A Hadoop 3.3.2 features
• supercedes: HADOOP-15620 Über-jira: S3A phase VI: Hadoop 3.3 features