[HDFS-8533] Mismatch in displaying the "MissingBlock" count in fsck and in other metric reports

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Duplicate
• Affects Version/s: 2.7.0
• Assignee: J.Andreina
• Reporter: J.Andreina
• Created: 04/Jun/15 10:26
• Updated: 03/Dec/18 21:44
• Resolved: 03/Dec/18 21:44

Description

Number of DN = 2
Step 1: Write a file with replication factor - 3 .
Step 2: Corrupt a replica in DN1
Step 3: DN2 is down.

Missing Block count in report is as follows
Fsck report : 0
Jmx, "dfsadmin -report" , UI, logs : 1

In fsck , only block whose replicas are all missed and not been corrupted are counted

if (totalReplicasPerBlock == 0 && !isCorrupt) {
        // If the block is corrupted, it means all its available replicas are
        // corrupted. We don't mark it as missing given these available replicas
        // might still be accessible as the block might be incorrectly marked as
        // corrupted by client machines.

While in other reports even if all the replicas are corrupted , block is been considered as missed.

Please provide your thoughts : can we make missing block count consistent across all the reports same as implemented for fsck?

Issue Links

• is duplicated by: HDFS-10213 there is a corrupt/missing block report, but fsck result is HEALTHY
• is related to: HDFS-13999 Bogus missing block warning if the file is under construction when NN starts