[HADOOP-13904] DynamoDBMetadataStore to handle DDB throttling failures through retry policy - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: HADOOP-13345
Fix Version/s: HADOOP-13345
Component/s: fs/s3
Labels:
None

Target Version/s:

HADOOP-13345

Description

When you overload DDB, you get error messages warning of throttling, as documented by AWS

Reduce load on DDB by doing a table lookup before the create, then, in table create/delete operations and in get/put actions, recognise the error codes and retry using an appropriate retry policy (exponential backoff + ultimate failure)

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

HADOOP-13904-HADOOP-13345.001.patch
19/Jan/17 02:33
15 kB
Aaron Fabbri
HADOOP-13904-HADOOP-13345.002.patch
02/Feb/17 07:21
24 kB
Aaron Fabbri
HADOOP-13904-HADOOP-13345.003.patch
16/Feb/17 01:54
23 kB
Aaron Fabbri
HADOOP-13904-HADOOP-13345.004.patch
20/Feb/17 21:33
23 kB
Aaron Fabbri
screenshot-1.png
16/Feb/17 01:27
130 kB
Aaron Fabbri

Issue Links

is cloned by

HADOOP-14087 S3A typo in pom.xml test exclusions

Resolved

is related to

HADOOP-13908 S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore

Resolved

relates to

HADOOP-13899 tune dynamodb client & tests

Resolved

Activity

People

Assignee:: Aaron Fabbri

Reporter:: Steve Loughran

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 14/Dec/16 10:57

Updated:: 20/Feb/17 23:59

Resolved:: 20/Feb/17 23:59