[HADOOP-3326] ReduceTask should not sleep for 200 ms while waiting for merge to finish

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.18.0
• Hadoop Flags: Reviewed
• Release Note: Changed fetchOutputs() so that LocalFSMerger and InMemFSMergeThread threads are spawned only once. The thread gets notified when something is ready for merge. The merge happens when thresholds are met.
• Assignee: Sharad Agarwal
• Reporter: Owen O'Malley
• Created: 29/Apr/08 20:02
• Updated: 08/Jul/09 16:52
• Resolved: 05/Jun/08 21:10

Description

Currently the merge code in Reduce task does:

            // Wait for the on-disk merge to complete
            while (localFSMergeInProgress) {
              Thread.sleep(200);
            }
            
            //wait for an ongoing merge (if it is in flight) to complete
            while (mergeInProgress) {
              Thread.sleep(200);
            }