[HADOOP-13908] S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: HADOOP-13345
• Fix Version/s: HADOOP-13345
• Component/s: fs/s3
• Target Version/s: HADOOP-13345
• Hadoop Flags: Reviewed
• Assignee: Mingliang Liu
• Reporter: Mingliang Liu
• Created: 15/Dec/16 00:20
• Updated: 10/Jan/17 00:12
• Resolved: 10/Jan/17 00:08

Description

This was based on discussion in HADOOP-13455. Though we should not create table unless the config fs.s3a.s3guard.ddb.table.create is set true, we still have to get the existing table in DynamoDBMetadataStore#initialize() and wait for its becoming active, before any table/item operations.

Issue Links

• is depended upon by: HADOOP-13959 S3guard: replace dynamo.describe() call in init with more efficient query
• is duplicated by: HADOOP-13925 S3Guard: NPE when table is already populated in dynamodb and user specifies "fs.s3a.s3guard.ddb.table.create=false"
• relates to: HADOOP-13455 S3Guard: Write end user docs, change table autocreate default.
• relates to: HADOOP-13904 DynamoDBMetadataStore to handle DDB throttling failures through retry policy