[HADOOP-3130] Shuffling takes too long to get the last map output.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.18.0
• Hadoop Flags: Reviewed
• Assignee: Amar Kamat
• Reporter: Runping Qi
• Created: 29/Mar/08 00:37
• Updated: 22/Aug/08 19:50
• Resolved: 16/Apr/08 13:43

Description

I noticed that towards the end of shufflling, the map output fetcher of the reducer backs off too aggressively.
I attach a fraction of one reduce log of my job.
Noticed that the last map output was not fetched in 2 minutes.