[HDFS-16964] Improve processing of excess redundancy after failover

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: block placement
• Labels: pull-request-available
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Shuyan Zhang
• Reporter: Shuyan Zhang
• Created: 24/Mar/23 10:22
• Updated: 28/Jan/24 07:11
• Resolved: 28/Mar/23 08:18

Description

After failover, the block with excess redundancy cannot be processed until all replicas are not stale, because the stale ones may have been deleted. That is to say, we need to wait for the FBRs of all datanodes on which the block resides before deleting the redundant replicas. This is unnecessary, we can bypass stale replicas when dealing with excess replicas, and delete non-stale excess replicas in a more timely manner.

Issue Links

• links to: GitHub Pull Request #5510