Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4979

Capacity Scheduler does not always return no task to a TT if a job's memry requirements are not met

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      As per HADOOP-4035, the Capacity Scheduler should return no task to a TT if a job's high mem requirements are not met. This doesn't always happen. In the Scheduler's assignTasks() method, if a job's map task does not enough memory to run, the Scheduler looks at reduce tasks, and vice-versa. This can result in a case where a reduce task from another job is returned to the TT (if the high-mem job does not have a reduce task to run, for example), thus starving the high-mem job.

        Attachments

        1. 4979.3.patch
          8 kB
          Hemanth Yamijala
        2. 4979.2.patch
          8 kB
          Vivek Ratan
        3. 4979.1.patch
          8 kB
          Vivek Ratan

          Activity

            People

            • Assignee:
              vivekr Vivek Ratan
              Reporter:
              vivekr Vivek Ratan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: