[HDFS-14852] Removing from LowRedundancyBlocks does not remove the block from all queues

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0, 3.0.3, 3.1.2, 3.3.0
• Fix Version/s: 3.2.2, 3.3.1, 3.4.0
• Component/s: namenode
• Assignee: Hui Fei
• Reporter: Hui Fei
• Created: 18/Sep/19 11:55
• Updated: 10/Jun/21 07:44
• Resolved: 25/Aug/20 15:16

Description

LowRedundancyBlocks.java

// Some comments here
    if(priLevel >= 0 && priLevel < LEVEL
        && priorityQueues.get(priLevel).remove(block)) {
      NameNode.blockStateChangeLog.debug(
          "BLOCK* NameSystem.LowRedundancyBlock.remove: Removing block {}"
              + " from priority queue {}",
          block, priLevel);
      decrementBlockStat(block, priLevel, oldExpectedReplicas);
      return true;
    } else {
      // Try to remove the block from all queues if the block was
      // not found in the queue for the given priority level.
      for (int i = 0; i < LEVEL; i++) {
        if (i != priLevel && priorityQueues.get(i).remove(block)) {
          NameNode.blockStateChangeLog.debug(
              "BLOCK* NameSystem.LowRedundancyBlock.remove: Removing block" +
                  " {} from priority queue {}", block, i);
          decrementBlockStat(block, i, oldExpectedReplicas);
          return true;
        }
      }
    }
    return false;
  }

Source code is above, the comments as follow

// Try to remove the block from all queues if the block was
// not found in the queue for the given priority level.

The function "remove" does NOT remove the block from all queues.

Function add from LowRedundancyBlocks.java is used on some places and maybe one block in two or more queues.
We found that corrupt blocks mismatch corrupt files on NN web UI. Maybe it is related to this.

Upload initial patch