Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
-
Reviewed
-
Changed fetchOutputs() so that LocalFSMerger and InMemFSMergeThread threads are spawned only once. The thread gets notified when something is ready for merge. The merge happens when thresholds are met.
Description
Currently the merge code in Reduce task does:
// Wait for the on-disk merge to complete while (localFSMergeInProgress) { Thread.sleep(200); } //wait for an ongoing merge (if it is in flight) to complete while (mergeInProgress) { Thread.sleep(200); }