[HDFS-8818] Allow Balancer to run faster

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
• Component/s: balancer & mover
• Hadoop Flags: Reviewed

Release Note

Add a new conf "dfs.balancer.max-size-to-move" so that Balancer.MAX_SIZE_TO_MOVE becomes configurable.

• Assignee: Tsz-wo Sze
• Reporter: Tsz-wo Sze
• Created: 23/Jul/15 22:36
• Updated: 02/Oct/19 17:12
• Resolved: 02/May/17 21:44

Description

The original design of Balancer is intentionally to make it run slowly so that the balancing activities won't affect the normal cluster activities and the running jobs.

There are new use case that cluster admin may choose to balance the cluster when the cluster load is low, or in a maintain window. So that we should have an option to allow Balancer to run faster.

Issue Links

• breaks: HDFS-11742 Improve balancer usability after HDFS-8818
• breaks: HDFS-9354 Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows
• incorporates: HDFS-10171 Balancer should log config values
• relates to: HDFS-11377 Balancer hung due to no available mover threads