[HADOOP-6439] Shuffle deadlocks on wrong number of maps

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.21.0, 0.22.0
• Fix Version/s: 0.21.0
• Component/s: conf
• Hadoop Flags: Reviewed
• Assignee: V.V.Chaitanya Krishna
• Reporter: Owen O'Malley
• Created: 01/Dec/09 01:05
• Updated: 24/Aug/10 20:41
• Resolved: 21/Apr/10 19:38

Description

The new shuffle assumes that the number of maps is correct. The new JobSubmitter sets the old value. Something misfires in the middle causing:

09/12/01 00:00:15 WARN conf.Configuration: mapred.job.split.file is deprecated. Instead, use mapreduce.job.splitfile
09/12/01 00:00:15 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps

But my reduces got stuck at 2 maps / 12 when there were only 2 maps in the job.

Issue Links

• is related to: HADOOP-6105 Provide a way to automatically handle backward compatibility of deprecated keys