Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15619 Über-JIRA: S3Guard Phase IV: Hadoop 3.3 features
  3. HADOOP-16349

DynamoDBMetadataStore.getVersionMarkerItem() to log at info/warn on retry

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.2.0
    • Fix Version/s: None
    • Component/s: fs/s3
    • Labels:
      None

      Description

      If you delete the version marker from a S3Guard table, it appears to hang for 5 minutes.

      Only if you restart and turn logging to debug do you see that DynamoDBMetadataStore.getVersionMarkerItem() is sleeping and retrying.

      1. log at warn
      2. add entry to troubleshooting doc on the topic

      The cause of the failure can be any of

      • table being inited elsewhere: expectation, fast recovery
      • it's not a S3Guard table: it won't recover
      • it's a S3Guard table without a version marker: it won't recover.

      + consider having a shorter retry lifespan, though if it adds a new config point I'm a bit reluctant. For s3guard bucket-info it would make sense to change the policy to be aggressively short lived

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gabor.bota Gabor Bota
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: