[HDFS-17094] EC: Fix bug in block recovery when there are stale datanodes

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: erasure-coding
• Labels: pull-request-available
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Shuyan Zhang
• Reporter: Shuyan Zhang
• Created: 18/Jul/23 10:22
• Updated: 28/Jan/24 01:53
• Resolved: 20/Jul/23 01:22

Description

When a block recovery occurs, `RecoveryTaskStriped` in datanode expects `rBlock.getLocations()` and `rBlock. getBlockIndices()` to be in one-to-one correspondence. However, if there are locations in stale state when NameNode handles heartbeat, this correspondence will be disrupted. In detail, there is no stale location in `recoveryLocations`, but the block indices array is still complete (i.e. contains the indices of all the locations). This will cause `BlockRecoveryWorker.RecoveryTaskStriped#recover` to generate a wrong internal block ID, and the corresponding datanode cannot find the replica, thus making the recovery process fail. This bug needs to be fixed.

Issue Links

• links to: GitHub Pull Request #5854