Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2512

wait(5000) and notify() mechanism can be implemented instead of sleep(5000) in reduce task when there are no copies in progress and no new copies to schedule

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.20.2
    • Fix Version/s: None
    • Component/s: task
    • Labels:
      None

      Description

      ReduceTask.java
       
             try { 
                  if (numInFlight == 0 && numScheduled == 0) { 
                    // we should indicate progress as we don't want TT to think 
                    // we're stuck and kill us 
                    reporter.progress(); 
                    Thread.sleep(5000); 
                  } 
                } catch (InterruptedException e) { } // IGNORE 
      

      Here if we have no copies in flight and we can't schedule anything new, it is going to wait for 5000 millis. Instead of waiting for 5000 millis, this thread can wait with timeout and GetMapEventsThread can notify it if gets new map completion events earlier than 5000 millis time.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              devaraj.k Devaraj K
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: