Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-17002

Erasure coding:Generate parity blocks in time to prevent file corruption

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Not A Problem
    • 3.4.0
    • None
    • erasure-coding
    • None

    Description

      In current EC implementation, the corrupted parity block will not be regenerated in time. 

      Think about below scene when using RS-6-3-1024k EC policy:

      If three parity blocks p1, p2, p3 are all corrupted or deleted, we are not aware of it.

      Unfortunately, a data block is also corrupted in this time period,  then this file will be corrupted and can not be read by decoding.

       

      So, here we should always re-generate parity block in time when it is unhealthy.

       

      Attachments

        Activity

          People

            zhanghaobo farmmamba
            zhanghaobo farmmamba
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: