[HDFS-16057] Make sure the order for location in ENTERING_MAINTENANCE state

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.4.0, 3.3.2
• Fix Version/s: 3.4.0, 3.3.2
• Component/s: block placement
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.3.2
• Hadoop Flags: Reviewed
• Assignee: Tao Li
• Reporter: Tao Li
• Created: 08/Jun/21 09:10
• Updated: 11/Feb/24 14:17
• Resolved: 11/Jun/21 08:42

Description

We use comparator to sort locations in getBlockLocations(), and the expected result is: live -> stale -> entering_maintenance -> decommissioned.

But the networktopology. SortByDistance() will disrupt the order. We should also filtered out node in sate  AdminStates.ENTERING_MAINTENANCE before networktopology. SortByDistance().

 

org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager#sortLocatedBlock()

DatanodeInfoWithStorage[] di = lb.getLocations();
// Move decommissioned/stale datanodes to the bottom
Arrays.sort(di, comparator);

// Sort nodes by network distance only for located blocks
int lastActiveIndex = di.length - 1;
while (lastActiveIndex > 0 && isInactive(di[lastActiveIndex])) {
  --lastActiveIndex;
}
int activeLen = lastActiveIndex + 1;
if(nonDatanodeReader) {
  networktopology.sortByDistanceUsingNetworkLocation(client,
      lb.getLocations(), activeLen, createSecondaryNodeSorter());
} else {
  networktopology.sortByDistance(client, lb.getLocations(), activeLen,
      createSecondaryNodeSorter());
}

 

Issue Links

• links to: GitHub Pull Request #3084