[HADOOP-15947] Fix ITestDynamoDBMetadataStore test error issues

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.3.0
• Component/s: fs/s3
• Assignee: Gabor Bota
• Reporter: Gabor Bota
• Created: 25/Nov/18 08:11
• Updated: 13/Mar/19 18:08
• Resolved: 28/Nov/18 17:53

Description

When running regression hadoop-aws integration tests for HADOOP-15370, I got the following errors in ITestDynamoDBMetadataStore:

[ERROR] Tests run: 40, Failures: 4, Errors: 2, Skipped: 0, Time elapsed: 177.303 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore
[ERROR] testBatchWrite(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore)  Time elapsed: 1.262 s  <<< ERROR!
java.lang.NullPointerException
	at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore.doTestBatchWrite(ITestDynamoDBMetadataStore.java:365) (...)

[ERROR] testProvisionTable(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore)  Time elapsed: 11.394 s  <<< FAILURE!
java.lang.AssertionError: expected:<20> but was:<10> (...)
org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore.testProvisionTable(ITestDynamoDBMetadataStore.java:594) (...)

[ERROR] testPruneUnsetsAuthoritative(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore)  Time elapsed: 2.323 s  <<< ERROR!
java.lang.NullPointerException
	at org.apache.hadoop.fs.s3a.s3guard.MetadataStoreTestBase.testPruneUnsetsAuthoritative(MetadataStoreTestBase.java:737) (...)

[ERROR] testDeleteSubtree(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore)  Time elapsed: 2.377 s  <<< FAILURE!
java.lang.AssertionError: Directory /ADirectory2 is null in cache (...)

[ERROR] testPutNew(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore)  Time elapsed: 1.466 s  <<< FAILURE!
java.lang.AssertionError: Directory /da2 is null in cache (...)

[ERROR] testDeleteSubtreeHostPath(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore)  Time elapsed: 2.378 s  <<< FAILURE!
java.lang.AssertionError: Directory s3a://cloudera-dev-gabor-ireland/ADirectory2 is null in cache (...)

I create this jira to handle and fix all of these issues.

Issue Links

• breaks: HADOOP-15827 NPE in DynamoDBMetadataStore.lambda$listChildren for root + auth S3Guard
• breaks: HADOOP-16186 S3Guard: NPE in DynamoDBMetadataStore.lambda$listChildren