[HADOOP-15888] ITestDynamoDBMetadataStore can leak (large) DDB tables in test failures/timeout

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.1.2
• Component/s: fs/s3, test
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 30/Oct/18 17:37
• Updated: 07/Jun/19 17:30
• Resolved: 07/Jun/19 17:30

Description

This is me doing some backporting of patches from branch-3.2, so it may be an intermediate condition but

1. I'd noticed I wasn't actually running ITestDynamoDBMetadataStore
2. so I set it up to work with teh right config opts (table and region)
3. but the tests were timing out
4. looking at DDB tables in the AWS console showed a number of DDB tables "testProvisionTable", "testProvisionTable", created, each with "500 read, 100 write capacity (i.e. ~$50/month)

I haven't replicated this in trunk/branch-3.2 itself, but its clearly dangerous. At the very least, we should have a size of 1 R/W in all creations, so the cost of a test failure is neglible, and then we should document the risk and best practise.

Also: use "s3guard" as the table prefix to make clear its origin

Issue Links

• depends upon: HADOOP-15563 S3Guard to support creating on-demand DDB tables
• Is contained by: HADOOP-15563 S3Guard to support creating on-demand DDB tables