[HDFS-17002] Erasure coding:Generate parity blocks in time to prevent file corruption

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 3.4.0
• Component/s: erasure-coding
• Assignee: farmmamba
• Reporter: farmmamba
• Created: 08/May/23 08:27
• Updated: 11/May/23 01:56
• Resolved: 11/May/23 01:55

Description

In current EC implementation, the corrupted parity block will not be regenerated in time. 

Think about below scene when using RS-6-3-1024k EC policy:

If three parity blocks p1, p2, p3 are all corrupted or deleted, we are not aware of it.

Unfortunately, a data block is also corrupted in this time period,  then this file will be corrupted and can not be read by decoding.

 

So, here we should always re-generate parity block in time when it is unhealthy.