Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13345 S3Guard: Improved Consistency for S3A
  3. HADOOP-13651

S3Guard: S3AFileSystem Integration with MetadataStore

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • HADOOP-13345
    • fs/s3
    • None

    Description

      Modify S3AFileSystem et al. to optionally use a MetadataStore for metadata consistency and caching.

      Implementation should have minimal overhead when no MetadataStore is configured.

      Attachments

        1. HADOOP-13651-HADOOP-13345.001.patch
          59 kB
          Aaron Fabbri
        2. HADOOP-13651-HADOOP-13345.002.patch
          90 kB
          Aaron Fabbri
        3. HADOOP-13651-HADOOP-13345.003.patch
          96 kB
          Aaron Fabbri
        4. HADOOP-13651-HADOOP-13345.004.patch
          98 kB
          Aaron Fabbri
        5. HADOOP-13651-HADOOP-13345.005.patch
          99 kB
          Aaron Fabbri
        6. HADOOP-13651-HADOOP-13345.006.patch
          99 kB
          Aaron Fabbri
        7. v6-test-report.txt
          7 kB
          Aaron Fabbri

        Issue Links

          depends upon

          Sub-task - The sub-task of the issue HADOOP-13631 S3Guard: implement move() for LocalMetadataStore, add unit tests

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Sub-task - The sub-task of the issue HADOOP-13761 S3Guard: implement retries for DDB failures and throttling; translate exceptions

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-13760 S3Guard: add delete tracking

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-13793 s3guard: add inconsistency injection, integration tests

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              fabbri Aaron Fabbri
              fabbri Aaron Fabbri
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: