[HADOOP-16384] S3A: Avoid inconsistencies between DDB and S3

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.0
• Component/s: fs/s3
• Target Version/s: 3.3.0
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 20/Jun/19 16:27
• Updated: 31/Jul/19 18:17
• Resolved: 12/Jul/19 12:02

Description

HADOOP-15183 added detection and rejection of prune updates when the store is inconsistent (i.e. when it tries to update an entry twice in the same operation, the second time with one that is inconsistent with the first)

Now that we can detect this, we should address it. We are lucky here in that my DDB table is currently inconsistent: prune is failing.

Plan

1. new test to run in the sequential phase, which does a s3guard prune against the bucket used in tests
2. use this to identify/debug the issue
3. replicate the problem in the ITestDDBMetastore tests
4. decide what to do in this world. Tell the user to run fsck? skip?

Issue Links

• causes: HADOOP-16481 ITestS3GuardDDBRootOperations.test_300_MetastorePrune needs to set region
• contains: HADOOP-16397 Hadoop S3Guard Prune command to support a -tombstone option.
• contains: HADOOP-16406 ITestDynamoDBMetadataStore.testProvisionTable times out intermittently
• is related to: HADOOP-16380 S3A tombstones can confuse empty directory status
• relates to: HADOOP-16396 Allow authoritative mode on a subdirectory
• links to: GitHub Pull Request #1003