[HDFS-10297] Increase default balance bandwidth and concurrent moves

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: balancer & mover
• Labels: supportability
• Target Version/s: 2.8.0
• Assignee: John Zhuge
• Reporter: John Zhuge
• Created: 15/Apr/16 05:25
• Updated: 30/Aug/16 01:16
• Resolved: 28/Apr/16 00:04

Description

Adjust the default values to better support the current level of customer host and network configurations.

Increase the default for property dfs.datanode.balance.bandwidthPerSec from 1 to 10 MB. Apply to DN. 10 MB/s is about 10% of the GbE network.

Increase the default for property dfs.datanode.balance.max.concurrent.moves from 5 to 50. Apply to DN and Balancer. The default number of DN receiver threads is 4096. The default number of balancer mover threads is 1000.