Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4749

Killing multiple attempts of a task taker longer as more attempts are killed

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.1.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The following was noticed on a mr job running on hadoop 1.1.0

      1. Start an mr job with 1 mapper

      2. Wait for a min

      3. Kill the first attempt of the mapper and then subsequently kill the other 3 attempts in order to fail the job

      The time taken to kill the task grew exponentially.

      1st attempt was killed immediately.
      2nd attempt took a little over a min
      3rd attempt took approx. 20 mins
      4th attempt took around 3 hrs.

      The command used to kill the attempt was "hadoop job -fail-task"

      Note that the command returned immediately as soon as the fail attempt was accepted but the time the attempt was actually killed was as stated above.

      1. MAPREDUCE-4749.branch-1.patch
        0.5 kB
        Arpit Gupta
      2. MAPREDUCE-4749.branch-1.patch
        6 kB
        Arpit Gupta
      3. MAPREDUCE-4749.branch-1.patch
        22 kB
        Arpit Gupta
      4. MAPREDUCE-4749.branch-1.patch
        22 kB
        Arpit Gupta
      5. MAPREDUCE-4749.branch-1.patch
        22 kB
        Arpit Gupta
      6. MAPREDUCE-4749.branch-1.patch
        22 kB
        Arpit Gupta

        Issue Links

          Activity

            People

            • Assignee:
              Arpit Gupta
              Reporter:
              Arpit Gupta
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development