Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15038

Abstract MetadataStore in S3Guard into a common module.

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.2.0, 2.9.2, 3.0.3, 3.1.2
    • Fix Version/s: None
    • Component/s: fs
    • Labels:
      None

      Description

      Open this JIRA to discuss if we should move MetadataStore in S3Guard into a common module.

      Based on this work, other filesystem or object store can implement their own metastore for optimization (known issues like consistency problem and metadata operation performance). Steve Loughran and other guys have done many base and great works in S3Guard. It is very helpful to start work. I did some perf test in HADOOP-14098, and started related work for Aliyun OSS. Indeed there are still works to do for S3Guard, like metadata cache inconsistent with S3 and so on. It also will be a problem for other object store. However, we can do these works in parallel.

      Steve Loughran Aaron Fabbri Kai Zheng Any suggestion is appreciated.

        Attachments

        1. HADOOP-15038.001.patch
          88 kB
          wujinhu

          Issue Links

            Activity

              People

              • Assignee:
                wujinhu wujinhu
                Reporter:
                uncleGen Genmao Yu
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated: