Description
Scenario: customer wants to only give a Hadoop cluster access to a subtree of an S3 bucket.
For example, assume Hadoop uses some IAM identity "hadoop", which they wish to grant full permission to everything under the following path:
s3a://bucket/a/b/c/hadoop-dir
they don't want hadoop user to be able to read/list/delete anything outside of the hadoop-dir "subdir"
Problems:
To implement the "directory structure on flat key space" emulation logic we use to present a Hadoop FS on top of a blob store, we need to create / delete / list ancestors of hadoop-dir. (to maintain the invariants (1) zero-byte object with key ending in '/' exists iff empty directory is there and (2) files cannot live beneath files, only directories.)
I'd like us to (1) document a an example with IAM ACLs policies that gets this basic functionality, and consider (2) making improvements to make this easier.
We've discussed some of these issues before but I didn't see a dedicated JIRA.
Attachments
Issue Links
- is related to
-
HADOOP-15460 S3A FS to add "fs.s3a.create.performance" to the builder file creation option set
-
- Resolved
-
- relates to
-
-
- Resolved
-
-
HADOOP-15542 S3AFileSystem - FileAlreadyExistsException when prefix is a file and part of a directory tree
-
- Resolved
-