Scenario: customer wants to only give a Hadoop cluster access to a subtree of an S3 bucket.
For example, assume Hadoop uses some IAM identity "hadoop", which they wish to grant full permission to everything under the following path:
they don't want hadoop user to be able to read/list/delete anything outside of the hadoop-dir "subdir"
To implement the "directory structure on flat key space" emulation logic we use to present a Hadoop FS on top of a blob store, we need to create / delete / list ancestors of hadoop-dir. (to maintain the invariants (1) zero-byte object with key ending in '/' exists iff empty directory is there and (2) files cannot live beneath files, only directories.)
I'd like us to (1) document a an example with IAM ACLs policies that gets this basic functionality, and consider (2) making improvements to make this easier.
We've discussed some of these issues before but I didn't see a dedicated JIRA.