Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14825 Über-JIRA: S3Guard Phase II: Hadoop 3.1 features
  3. HADOOP-14220

Enhance S3GuardTool with bucket-info and set-capacity commands, tests

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-beta1
    • 2.9.0, 3.0.0-beta1
    • fs/s3
    • None

    Description

      Add a diagnostics command to s3guard which does whatever we need to diagnose problems for a specific (named) s3a url. This is something which can be attached to bug reports as well as used by developers.

      • Properties to log (with provenance attribute, which can track bucket overrides: s3guard metastore setup, autocreate, capacity,....
      • table present/absent
      • # of keys in DDB table for that bucket?
      • any other stats?

      Attachments

        1. HADOOP-14220-006.patch
          84 kB
          Steve Loughran
        2. HADOOP-14220-008.patch
          84 kB
          Steve Loughran
        3. HADOOP-14220-009.patch
          89 kB
          Steve Loughran
        4. HADOOP-14220-010.patch
          89 kB
          Steve Loughran
        5. HADOOP-14220-012.patch
          89 kB
          Steve Loughran
        6. HADOOP-14220-013.patch
          89 kB
          Steve Loughran
        7. HADOOP-14220-013.patch
          88 kB
          Aaron Fabbri
        8. HADOOP-14220-014.patch
          89 kB
          Steve Loughran
        9. HADOOP-14220-015.patch
          88 kB
          Steve Loughran
        10. HADOOP-14220-016.patch
          88 kB
          Aaron Fabbri
        11. HADOOP-14220-branch-2-016.patch
          119 kB
          Steve Loughran
        12. HADOOP-14220-branch-2-017.patch
          88 kB
          Steve Loughran
        13. HADOOP-14220-HADOOP-13345-001.patch
          9 kB
          Steve Loughran
        14. HADOOP-14220-HADOOP-13345-002.patch
          41 kB
          Steve Loughran
        15. HADOOP-14220-HADOOP-13345-003.patch
          62 kB
          Steve Loughran
        16. HADOOP-14220-HADOOP-13345-004.patch
          77 kB
          Steve Loughran
        17. HADOOP-14220-HADOOP-13345-005.patch
          82 kB
          Steve Loughran

        Issue Links

          incorporates

          Sub-task - The sub-task of the issue HADOOP-14345 S3Guard: S3GuardTool to support provisioning existing metadata store

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-14800 eliminate double stack trace on some s3guard CLI failures

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-14801 s3guard diff demand creates a new table

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is duplicated by

          Sub-task - The sub-task of the issue HADOOP-14007 cli to list info about a bucket (S3guard or not)

          • Major - Major loss of function.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue HADOOP-14158 Possible for modified configuration to leak into metadatastore in S3GuardTool

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-14759 S3GuardTool prune to prune specific bucket entries

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-14851 LambdaTestUtils.eventually() doesn't spin on Assertion failures

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: