[HADOOP-15176] Enhance IAM Assumed Role support in S3A client

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.1.0
• Component/s: fs/s3, test
• Target Version/s: 3.1.0
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 17/Jan/18 13:49
• Updated: 09/Jun/18 13:08
• Resolved: 15/Feb/18 15:59

Description

Followup HADOOP-15141 with

• Code to generate basic AWS json policies somewhat declaratively (no hand coded strings)
• Tests to simulate users with different permissions down the path of a single bucket
• test-driven changes to S3A client to handle user without full write up the FS tree
• move the new authenticator into the s3a sub-package "auth", where we can put more auth stuff (that base s3a package is getting way too big)

Issue Links

• is depended upon by: HADOOP-15183 S3Guard store becomes inconsistent after partial failure of rename
• is related to: HADOOP-15525 s3a: clarify / improve support for mixed ACL buckets
• is related to: HADOOP-13278 S3AFileSystem mkdirs does not need to validate parent path components
• relates to: HADOOP-15141 Support IAM Assumed roles in S3A