[HDFS-1765] Block Replication should respect under-replication block priority

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.0
• Fix Version/s: 2.0.0-alpha, 0.23.7
• Component/s: namenode
• Target Version/s: 0.23.3
• Hadoop Flags: Reviewed
• Assignee: Uma Maheswara Rao G
• Reporter: Hairong Kuang
• Created: 17/Mar/11 19:01
• Updated: 21/Sep/23 17:39
• Resolved: 07/Mar/12 07:06

Description

Currently under-replicated blocks are assigned different priorities depending on how many replicas a block has. However the replication monitor works on blocks in a round-robin fashion. So the newly added high priority blocks won't get replicated until all low-priority blocks are done. One example is that on decommissioning datanode WebUI we often observe that "blocks with only decommissioning replicas" do not get scheduled to replicate before other blocks, so risking data availability if the node is shutdown for repair before decommission completes.

Issue Links

• is related to: HDFS-4366 Block Replication Policy Implementation May Skip Higher-Priority Blocks for Lower-Priority Blocks