[MAPREDUCE-4299] Terasort hangs with MR2 FifoScheduler

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0-alpha
• Fix Version/s: 0.23.3, 2.0.2-alpha
• Component/s: mrv2
• Assignee: Thomas White
• Reporter: Thomas White
• Created: 31/May/12 20:41
• Updated: 11/Oct/12 17:48
• Resolved: 13/Jul/12 20:56

Description

What happens is that the number of reducers ramp up until they occupy all of the job's containers, at which point the maps no longer make any progress and the job hangs.

When the same job is run with the CapacityScheduler it succeeds, so this looks like a FifoScheduler bug.

Issue Links

• is duplicated by: MAPREDUCE-4358 Reducers are assigned containers before all maps are assigned containers
• is duplicated by: MAPREDUCE-4613 Scheduling of reduce tasks results in starvation
• is related to: MAPREDUCE-4560 Job can get stuck in a deadlock between mappers and reducers for low values of mapreduce.job.reduce.slowstart.completedmaps (<<1)