[HADOOP-14810] S3Guard: handle provisioning failure through backoff & retry (& metrics)

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: HADOOP-13345
• Component/s: fs/s3
• Assignee: Unassigned
• Reporter: Steve Loughran
• Created: 29/Aug/17 10:48
• Updated: 01/Sep/17 13:40
• Resolved: 01/Sep/17 13:40

Description

S3Guard can't handle overloaded tables.

I think we all though the API did: it doesn't; exceptions get raised and the caller is expected to handle it.

This relates very much to the s3a-lambda invocation code in HADOOP-13786 to handle failures during commit, and the need for all the S3AFileSystem calls of the S3 APIs to handle transient failures like throttling, and again, needs some fault injection to verify the handling, metrics to count rate so it can be monitored & used to understand why work is underperforming.

Issue Links

• duplicates: HADOOP-13761 S3Guard: implement retries for DDB failures and throttling; translate exceptions