[HDFS-10316] revisit corrupt replicas count

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Walter Su
• Created: 20/Apr/16 05:32
• Updated: 25/Apr/16 04:03

Description

A DN has 4 types of storages:
1. NORMAL
2. READ_ONLY
3. FAILED
4. (missing/pruned)

blocksMap.numNodes(blk) counts 1,2,3
blocksMap.getStorages(blk) counts 1,2,3

countNodes(blk).corruptReplicas() counts 1,2
corruptReplicas counts 1,2,3,4. Because findAndMarkBlockAsCorrupt(..) supports adding blk to the map even if the storage is not found.

The inconsistency causes bugs like HDFS-9958.

Issue Links

• is related to: HDFS-9958 BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages.