Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8031 Follow-on work for erasure coding phase I (striping layout)
  3. HDFS-7859

Erasure Coding: Persist erasure coding policies in NameNode

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0-beta1
    • None
    • Incompatible change, Reviewed
    • Persist all built-in erasure coding policies and user defined erasure coding policies into NameNode fsImage and editlog reliably, so that all erasure coding policies remain consistent after NameNode restart.

    Description

      In meetup discussion with zhz and jingzhao, it's suggested that we persist EC schemas in NameNode centrally and reliably, so that EC zones can reference them by name efficiently.

      Attachments

        1. HDFS-7859.001.patch
          32 kB
          Xinwei Qin
        2. HDFS-7859.002.patch
          35 kB
          Xinwei Qin
        3. HDFS-7859-HDFS-7285.002.patch
          35 kB
          Allen Wittenauer
        4. HDFS-7859-HDFS-7285.002.patch
          35 kB
          Zhe Zhang
        5. HDFS-7859-HDFS-7285.003.patch
          27 kB
          Xinwei Qin
        6. HDFS-7859.004.patch
          29 kB
          Xinwei Qin
        7. HDFS-7859.005.patch
          36 kB
          Xinwei Qin
        8. HDFS-7859.006.patch
          36 kB
          Xinwei Qin
        9. HDFS-7859.007.patch
          34 kB
          Xinwei Qin
        10. HDFS-7859.008.patch
          36 kB
          Surendra Singh Lilhore
        11. HDFS-7859.009.patch
          88 kB
          Surendra Singh Lilhore
        12. HDFS-7859.010.patch
          86 kB
          Sammi Chen
        13. HDFS-7859.011.patch
          53 kB
          Sammi Chen
        14. HDFS-7859.012.patch
          66 kB
          Sammi Chen
        15. HDFS-7859.013.patch
          34 kB
          Sammi Chen
        16. HDFS-7859.014.patch
          24 kB
          Sammi Chen
        17. HDFS-7859.015.patch
          20 kB
          Sammi Chen
        18. HDFS-7859.016.patch
          19 kB
          Sammi Chen
        19. HDFS-7859.017.patch
          19 kB
          Sammi Chen
        20. HDFS-7859.018.patch
          19 kB
          Sammi Chen
        21. HDFS-7859.019.patch
          19 kB
          Sammi Chen

        Issue Links

          contains

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-12395 Support erasure coding policy operations in namenode edit log

          • Major - Major loss of function.
          • Resolved
          depends upon

          Sub-task - The sub-task of the issue HADOOP-11643 Define EC schema API for ErasureCodec

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Sub-task - The sub-task of the issue HDFS-11467 Support ErasureCoding section in OIV XML/ReverseXML

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Sub-task - The sub-task of the issue HDFS-7866 Erasure coding: NameNode manages multiple erasure coding policies

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDFS-8140 ECSchema supports for offline EditsVisitor over an OEV XML file

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDFS-8295 Add MODIFY and REMOVE ECSchema editlog operations

          • Major - Major loss of function.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue HDFS-10786 Erasure Coding: Add removeErasureCodingPolicy API

          • Major - Major loss of function.
          • Resolved
          is required by

          Sub-task - The sub-task of the issue HDFS-8023 Erasure Coding: retrieve eraure coding schema for a file from NameNode

          • Major - Major loss of function.
          • Resolved

          Task - A task that needs to be done. HDFS-8020 Erasure Coding: constructing and mapping BlockGroup info to utilize and call ErasureCoder API

          • Major - Major loss of function.
          • Patch Available

          Activity

            People

              Sammi Sammi Chen
              drankye Kai Zheng
              Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: