Über-jira: S3A features/fixes for Hadoop 3.4
As usual, this will clutter up with everything which hasn't gone in: don't interpret presence on this list as a commitment to implement.
And for anyone wanting to add patches
MUST
- reviews via github PRs
- no declaration of AWS S3 endpoint (or other S3 impl) -no review
SHOULD
- have a setup for testing SSE-KMS, DDB/S3Guard
- including an assumed role we can use for AssumedRole Delegation Tokens
If you are going near those bits of code, they uprate from SHOULD to MUST.
- depends upon
-
HADOOP-15387 Produce a shaded hadoop-cloud-storage JAR for applications to use
-
- Open
-
-
MAPREDUCE-7182 MapReduce input format/record readers to support S3 select queries
-
- Open
-
-
HADOOP-15788 Improve Distcp for long-haul/cloud deployments
-
- Open
-
-
HADOOP-16604 Provide copy functionality for cloud native applications
-
- Open
-
-
MAPREDUCE-6760 LocatedFileStatusFetcher to use listFiles(recursive)
-
- Open
-
-
HADOOP-13278 S3AFileSystem mkdirs does not need to validate parent path components
-
- Open
-
-
HADOOP-16844 FS/FC createFile() builder to add a lazy-check option for async probes of the store
-
- Open
-
-
HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden
-
- Resolved
-
-
MAPREDUCE-5907 Improve getSplits() performance for fs implementations that can utilize performance gains from recursive listing
-
- Patch Available
-
- is blocked by
-
HADOOP-16111 FileSystem.closeAllForUGI closes() filesystem in Sync block
-
- Open
-
-
HADOOP-15606 FSDataInputStream/FSInputStream to implement getFileLength()
-
- Open
-
- supercedes
-
HADOOP-15620 Über-jira: S3A phase VI: Hadoop 3.3 features
-
- Resolved
-