[HDFS-7281] Missing block is marked as corrupted block

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha1
• Fix Version/s: 3.0.0-alpha1
• Component/s: namenode
• Labels: supportability
• Hadoop Flags: Incompatible change, Reviewed

Release Note

The patch improves the reporting around missing blocks and corrupted blocks.
 
1. A block is missing if and only if all DNs of its expected replicas are dead.
2. A block is corrupted if and only if all its available replicas are corrupted. So if a block has 3 replicas; one of the DN is dead, the other two replicas are corrupted; it will be marked as corrupted.
3. A new line is added to fsck output to display the corrupt block size per file.
4. A new line is added to fsck output to display the number of missing blocks in the summary section.

• Assignee: Ming Ma
• Reporter: Ming Ma
• Created: 23/Oct/14 01:42
• Updated: 12/May/16 18:15
• Resolved: 01/May/15 16:00

Description

In the situation where the block lost all its replicas, fsck shows the block is missing as well as corrupted. Perhaps it is better not to mark the block corrupted in this case. The reason it is marked as corrupted is numCorruptNodes == numNodes == 0 in the following code.

BlockManager
    final boolean isCorrupt = numCorruptNodes == numNodes;

Would like to clarify if it is the intent to mark missing block as corrupted or it is just a bug.