Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7345

Local Repairable Codes (LRC)

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • None
    • None
    • erasure-coding
    • None

    Description

      HDFS-7285 proposes to support Erasure Coding inside HDFS, supports multiple Erasure Coding codecs via pluggable codec framework (HDFS-7337) and implements Reed Solomon code by default. This is to support a more advanced coding mechanism, Local Repairable Codes (LRC). As discussed in Xorbas Hadoop Project Page and the paper attached there, LRC can reduce the number of erasure coding chunks that need to be read when recovering missing data chunks, while still keeping the storage overhead low.

      Attachments

        1. LRC-compare.pdf
          211 kB
          Kai Zheng

        Issue Links

          depends upon

          New Feature - A new feature of the product, which has yet to be developed. HDFS-7337 Configurable and pluggable erasure codec and policy

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              hitliuyi Yi Liu
              drankye Kai Zheng
              Votes:
              0 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

                Created:
                Updated: