[HADOOP-299] maps from second jobs will not run until the first job finishes completely

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.3.2
• Fix Version/s: 0.4.0
• Assignee: Owen O'Malley
• Reporter: Owen O'Malley
• Created: 14/Jun/06 04:23
• Updated: 08/Jul/09 16:51
• Resolved: 20/Jun/06 02:03

Description

Because of the logic in the JobTracker's pollForNewTask, second jobs will rarely start running maps until the first job finishes completely. The JobTracker leaves room to re-run failed maps from the first job and it reserves the total number of maps for the first job. Thus, if you have more maps in the first job than your cluster capacity, none of the second job maps will ever run.

I propose setting the reserve to 1% of the first job's maps.

Issue Links

• duplicates: HADOOP-295 jobs don't get executed in parallel