[HDFS-8731] Erasure coding: For a small file missing and under replicated ec-block calculation is incorrect

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: J.Andreina
• Reporter: J.Andreina
• Created: 08/Jul/15 04:46
• Updated: 08/Jul/15 05:27
• Resolved: 08/Jul/15 05:27

Description

Write a small file (whose replica detail would be : one data replica and 3 parity replicas .)

Now :
totalReplicasPerBlock = 4
targetFileReplication = 9 (calculated based on default RS-6-3 schema)

Hence the small file ec-blocks is calculated as always missing and underreplicated. which is wrong...

targetFileReplication = (short) (ecSchema.getNumDataUnits() + ecSchema.getNumParityUnits());

      // count missing replicas / under replicated blocks
      if (totalReplicasPerBlock < targetFileReplication && totalReplicasPerBlock > 0) {
        res.missingReplicas += (targetFileReplication - totalReplicasPerBlock);
	res.numUnderReplicatedBlocks += 1;

Issue Links

• is related to: HDFS-8563 Erasure Coding: fsck handles file smaller than a full stripe