Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13345 S3Guard: Improved Consistency for S3A
  3. HADOOP-14810

S3Guard: handle provisioning failure through backoff & retry (& metrics)

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • HADOOP-13345
    • None
    • fs/s3
    • None

    Description

      S3Guard can't handle overloaded tables.

      I think we all though the API did: it doesn't; exceptions get raised and the caller is expected to handle it.

      This relates very much to the s3a-lambda invocation code in HADOOP-13786 to handle failures during commit, and the need for all the S3AFileSystem calls of the S3 APIs to handle transient failures like throttling, and again, needs some fault injection to verify the handling, metrics to count rate so it can be monitored & used to understand why work is underperforming.

      Attachments

        1. summary.txt
          21 kB
          Steve Loughran

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            stevel@apache.org Steve Loughran
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment