[HADOOP-13936] S3Guard: DynamoDB can go out of sync with S3AFileSystem.delete()

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Duplicate
• Affects Version/s: 3.0.0-beta1, 3.1.0, 3.1.1
• Component/s: fs/s3
• Assignee: Steve Loughran
• Reporter: Rajesh Balamohan
• Created: 23/Dec/16 00:13
• Updated: 12/Jun/19 08:50
• Resolved: 28/Mar/19 13:57

Description

As a part of S3AFileSystem.delete operation innerDelete is invoked, which deletes keys from S3 in batches (default is 1000). But DynamoDB is updated only at the end of this operation. This can cause issues when deleting large number of keys.

E.g, it is possible to get exception after deleting 1000 keys and in such cases dynamoDB would not be updated. This can cause DynamoDB to go out of sync.

Issue Links

• is depended upon by: HADOOP-14936 S3Guard: remove "experimental" from documentation
• is part of: HADOOP-15183 S3Guard store becomes inconsistent after partial failure of rename
• is related to: HADOOP-15621 S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing
• relates to: HADOOP-15191 Add Private/Unstable BulkDelete operations to supporting object stores for DistCP
• relates to: HADOOP-15193 add bulk delete call to metastore API & DDB impl
• relates to: HADOOP-13980 S3Guard CLI: Add fsck check and fix commands