[HADOOP-16474] S3Guard ProgressiveRenameTracker to mark dest dir as authoritative on success

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.3.0
• Component/s: fs/s3
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 30/Jul/19 16:10
• Updated: 10/Jan/20 11:15
• Resolved: 10/Jan/20 11:14

Description

After a directory rename is successful, the destination will contain only those files which have been copied by the S3guard-enabled client, with the directory tree updated as new entries are added.

At that point, the ProgressiveRenameTracker could tell the store to complete the rename and in so doing, give clients maximum performance without needing any LIST commands.

Issue Links

• Is contained by: HADOOP-16697 audit/tune s3a authoritative flag in s3guard DDB Table