[HADOOP-17261] s3a rename() now requires s3:deleteObjectVersion permission

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.3.1
• Component/s: fs/s3
• Labels: pull-request-available
• Target Version/s: 3.4.0
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 14/Sep/20 19:53
• Updated: 14/Oct/20 09:00
• Resolved: 14/Oct/20 09:00

Description

With the directory marker change (HADOOP-13230) you need the s3:deleteObjectVersion permission in your role, else the operation will fail in the bulk delete, if S3Guard is in use

Root cause
-if fileStatus has a versionId, we pass that in to the delete KeyVersion pair
-an unguarded listing doesn't get that versionId, so this is not an issue
-but if files in a directory were previously created such that S3Guard has their versionId in its tables, that is used in the request
-which then fails if the caller doesn't have the permission

Although we say "you need s3:delete*", this is a regression as any IAM role without the permission will have rename fail during delete

Issue Links

• is caused by: HADOOP-13230 S3A to optionally retain directory markers
• links to: GitHub Pull Request #2303