Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5337

JobTracker greedily schedules tasks without running tasks to join

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.0
    • Fix Version/s: 0.20.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This issue was observed when JobTracker was restarted 3 times and observed that 4 instances of each reduce task were running. This issue is observed when cluster is not fully occupied.

      In testcase: Map/reduces capacity is 200/200 slots respectively and Job profile is 11000 maps, 10 reduces and speculative execution is off. JobTracker was restarted 3 times in small intervals of about 5 mins and after recovery, 40 reduce tasks were running. Task details web page (taskdetails.jsp) was showing 4 running attempts of each reduce task.

        Attachments

        1. HADOOP-5337-v1.7.patch
          15 kB
          Amar Kamat
        2. HADOOP-5337-v1.9.patch
          16 kB
          Amar Kamat
        3. HADOOP-5337-v1.11.patch
          18 kB
          Amar Kamat
        4. HADOOP-5337-v1.12.patch
          18 kB
          Amar Kamat

          Activity

            People

            • Assignee:
              amar_kamat Amar Kamat
              Reporter:
              karams Karam Singh
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: