[MAPREDUCE-3641] CapacityScheduler should be more conservative assigning off-switch requests

• Type: Sub-task
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.23.0
• Fix Version/s: 0.23.1
• Component/s: mrv2, scheduler
• Target Version/s: 0.23.1
• Hadoop Flags: Reviewed
• Release Note: Making CapacityScheduler more conservative so as to assign only one off-switch container in a single scheduling iteration.
• Assignee: Arun C Murthy
• Reporter: Arun C Murthy
• Created: 08/Jan/12 06:14
• Updated: 05/Mar/12 02:49
• Resolved: 16/Jan/12 21:57

Description

In hadoop-1, the CS is very conservative handing out off-switch assignments, we need to do the same in YARN.

We noticed performance regressions due to this, particularly for reduces.

Issue Links

• blocks: MAPREDUCE-3525 Shuffle benchmark is nearly 1.5x slower in 0.23
• blocks: MAPREDUCE-3534 Compression benchmark run-time increased by 13% in 0.23