[HDFS-13910] Improve BlockPlacementPolicy for small clusters

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Duplicate
• Component/s: hdfs
• External issue URL: https://issues.apache.org/jira/browse/HDFS-13833
• Assignee: Shweta
• Reporter: Shweta
• Created: 11/Sep/18 20:57
• Updated: 11/Dec/18 01:18
• Resolved: 11/Sep/18 21:58

Description

From investigations and a few test occurrences, the NameNode BlockPlacementPolicy’s considerLoad can be bad for small test clusters.
A small (1-node) cluster may trigger a corner case of maxLoad = 0. In this case, filtering should not take place.

When there is no heartbeat, maxLoad will be 0 and the existing logic will filter out many nodes that have workload, which should be avoided.

Issue Links

• Blocked: HDFS-13833 Improve BlockPlacementPolicyDefault's consider load logic