Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4749

Killing multiple attempts of a task taker longer as more attempts are killed

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.1.0
    • 1.1.1
    • None
    • None
    • Reviewed

    Description

      The following was noticed on a mr job running on hadoop 1.1.0

      1. Start an mr job with 1 mapper

      2. Wait for a min

      3. Kill the first attempt of the mapper and then subsequently kill the other 3 attempts in order to fail the job

      The time taken to kill the task grew exponentially.

      1st attempt was killed immediately.
      2nd attempt took a little over a min
      3rd attempt took approx. 20 mins
      4th attempt took around 3 hrs.

      The command used to kill the attempt was "hadoop job -fail-task"

      Note that the command returned immediately as soon as the fail attempt was accepted but the time the attempt was actually killed was as stated above.

      Attachments

        1. MAPREDUCE-4749.branch-1.patch
          0.5 kB
          Arpit Gupta
        2. MAPREDUCE-4749.branch-1.patch
          6 kB
          Arpit Gupta
        3. MAPREDUCE-4749.branch-1.patch
          22 kB
          Arpit Gupta
        4. MAPREDUCE-4749.branch-1.patch
          22 kB
          Arpit Gupta
        5. MAPREDUCE-4749.branch-1.patch
          22 kB
          Arpit Gupta
        6. MAPREDUCE-4749.branch-1.patch
          22 kB
          Arpit Gupta

        Issue Links

          Activity

            People

              arpitgupta Arpit Gupta
              arpitgupta Arpit Gupta
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: