[HADOOP-16118] S3Guard to support on-demand DDB tables

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.0
• Component/s: fs/s3
• Labels: pull-request-available
• Target Version/s: 3.2.1
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 18/Feb/19 10:36
• Updated: 06/Apr/21 13:22
• Resolved: 03/Jun/19 12:23

Description

AWS now supports on demand DDB capacity

This has lowest cost and best scalability, so could be the default capacity. + add a new option to set-capacity.

Will depend on an SDK update: created HADOOP-16117.

Issue Links

• causes: HADOOP-16329 "Wrong Billing mode" test failure in ITestS3GuardToolDynamoDB
• depends upon: HADOOP-16117 Update AWS SDK to 1.11.563
• is depended upon by: HADOOP-14936 S3Guard: remove "experimental" from documentation
• is depended upon by: HADOOP-15563 S3Guard to support creating on-demand DDB tables
• is related to: HADOOP-15604 Bulk commits of S3A MPUs place needless excessive load on S3 & S3Guard
• links to: GitHub Pull Request #647
• links to: GitHub Pull Request #731