Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13345 S3Guard: Improved Consistency for S3A
  3. HADOOP-13904

DynamoDBMetadataStore to handle DDB throttling failures through retry policy

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: HADOOP-13345
    • Fix Version/s: HADOOP-13345
    • Component/s: fs/s3
    • Labels:
      None

      Description

      When you overload DDB, you get error messages warning of throttling, as documented by AWS

      Reduce load on DDB by doing a table lookup before the create, then, in table create/delete operations and in get/put actions, recognise the error codes and retry using an appropriate retry policy (exponential backoff + ultimate failure)

        Attachments

        1. HADOOP-13904-HADOOP-13345.001.patch
          15 kB
          Aaron Fabbri
        2. HADOOP-13904-HADOOP-13345.002.patch
          24 kB
          Aaron Fabbri
        3. HADOOP-13904-HADOOP-13345.003.patch
          23 kB
          Aaron Fabbri
        4. HADOOP-13904-HADOOP-13345.004.patch
          23 kB
          Aaron Fabbri
        5. screenshot-1.png
          130 kB
          Aaron Fabbri

          Issue Links

            Activity

              People

              • Assignee:
                fabbri Aaron Fabbri
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: