Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2177

The wait for spill completion should call Condition.awaitNanos(long nanosTimeout)

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.20.2
    • Fix Version/s: None
    • Component/s: tasktracker
    • Labels:
      None

      Description

      We sometimes saw maptask timeout in cdh3b2. Here is log from one of the maptasks:

      2010-11-04 10:34:23,820 INFO org.apache.hadoop.mapred.MapTask: Spilling map output: buffer full= true
      2010-11-04 10:34:23,820 INFO org.apache.hadoop.mapred.MapTask: bufstart = 119534169; bufend = 59763857; bufvoid = 298844160

      2010-11-04 10:34:23,820 INFO org.apache.hadoop.mapred.MapTask: kvstart = 438913; kvend = 585320; length = 983040
      2010-11-04 10:34:41,615 INFO org.apache.hadoop.mapred.MapTask: Finished spill 3
      2010-11-04 10:35:45,352 INFO org.apache.hadoop.mapred.MapTask: Spilling map output: buffer full= true

      2010-11-04 10:35:45,547 INFO org.apache.hadoop.mapred.MapTask: bufstart = 59763857; bufend = 298837899; bufvoid = 298844160
      2010-11-04 10:35:45,547 INFO org.apache.hadoop.mapred.MapTask: kvstart = 585320; kvend = 731585; length = 983040

      2010-11-04 10:45:41,289 INFO org.apache.hadoop.mapred.MapTask: Finished spill 4

      Note how long the last spill took.

      In MapTask.java, the following code waits for spill to finish:
      while (kvstart != kvend)

      { reporter.progress(); spillDone.await(); }

      In trunk code, code is similar.

      There is no timeout mechanism for Condition.await(). In case the SpillThread takes long before calling spillDone.signal(), we would see timeout.
      Condition.awaitNanos(long nanosTimeout) should be called.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                ted_yu Ted Yu
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated: