Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
3.4.0
-
Reviewed
-
Description
HADOOP-18995 upgrades the SDK version which allows connecting to a s3 express one zone support.
Complete support needs to be added to address tests that fail with s3 express one zone, additional tests, documentation etc.
- hadoop-common path capability to indicate that treewalking may encounter missing dirs
- use this in treewalking code in shell, mapreduce FileInputFormat etc to not fail during treewalks
- extra path capability for s3express too.
- tests for this
- anything else
A filesystem can now be probed for inconsistent directory listings through fs.hasPathCapability(path, "fs.capability.directory.listing.inconsistent")
If true, then treewalking code SHOULD NOT report a failure if, when walking into a subdirectory, a list/getFileStatus on that directory raises a FileNotFoundExceptin.
Attachments
Issue Links
- contains
-
HADOOP-18961 S3A: add s3guard command "bucket" to create buckets
-
- Resolved
-
-
HADOOP-18955 AWS SDK v2: add path capability probe "fs.s3a.capability.aws.v2"
-
- Resolved
-
- is a child of
-
HADOOP-18995 S3A: Upgrade AWS SDK version to 2.21.33 for Amazon S3 Express One Zone support
-
- Resolved
-
- is depended upon by
-
SPARK-47008 Spark to support S3 Express One Zone Storage
-
- Open
-
- is related to
-
HDDS-9815 Support S3 OneZone / S3 Express API changes for Ozone
-
- Open
-
- links to