[HDFS-13846] Safe blocks counter is not decremented correctly if the block is striped

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.2.0
• Component/s: hdfs
• Target Version/s: 3.2.0
• Hadoop Flags: Reviewed
• Assignee: Kitti Nanasi
• Reporter: Kitti Nanasi
• Created: 22/Aug/18 15:09
• Updated: 12/Sep/18 21:01
• Resolved: 12/Sep/18 18:25

Description

In BlockManagerSafeMode class, the "safe blocks" counter is incremented if the number of nodes containing the block equals to the number of data units specified by the erasure coding policy, which looks like this in the code:

final int safe = storedBlock.isStriped() ?
        ((BlockInfoStriped)storedBlock).getRealDataBlockNum() : safeReplication;
    if (storageNum == safe) {
      this.blockSafe++;

But when it is decremented the code does not check if the block is striped or not, just compares the number of nodes containing the block with 0 (safeReplication - 1) if the block is complete, which is not correct.

if (storedBlock.isComplete() &&
        blockManager.countNodes(b).liveReplicas() == safeReplication - 1) {
      this.blockSafe--;
      assert blockSafe >= 0;
      checkSafeMode();
    }