[HADOOP-3940] Reduce often attempts in memory merge with no work

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.18.0
• Fix Version/s: 0.18.1
• Hadoop Flags: Reviewed
• Assignee: Christopher Douglas
• Reporter: Christopher Douglas
• Created: 12/Aug/08 20:58
• Updated: 08/Jul/09 16:52
• Resolved: 13/Aug/08 22:38

Description

ReduceTask.ReduceCopier.ShuffleRamManager initializes numRequiredMapOutputs to 0, so one of the predicates in ShuffleRamManager::waitForDataToMerge, numPendingRequests < numRequiredMapOutputs, is false until the first map output is fetched and false again after the last map output is fetched. The InMemFSMergeThread thread will loop busily in this state.