[MAPREDUCE-1906] Lower default minimum heartbeat interval for tasktracker > Jobtracker

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.20.1, 0.20.2, 1.2.0
• Fix Version/s: 0.23.0, 1.1.0
• Component/s: jobtracker, performance, tasktracker
• Hadoop Flags: Reviewed
• Release Note: The default minimum heartbeat interval has been dropped from 3 seconds to 300ms to increase scheduling throughput on small clusters. Users may tune mapreduce.jobtracker.heartbeats.in.second to adjust this value.
• Assignee: Todd Lipcon
• Reporter: Scott Carey
• Created: 01/Jul/10 17:09
• Updated: 25/Sep/12 17:12
• Resolved: 31/Dec/10 18:16

Description

I get a 0% to 15% performance increase for smaller clusters by making the heartbeat throttle stop penalizing clusters with less than 300 nodes.

Between 0.19 and 0.20, the default minimum heartbeat interval increased from 2s to 3s. If a JobTracker is throttled at 100 heartbeats / sec for large clusters, why should a cluster with 10 nodes be throttled to 3.3 heartbeats per second?