Hadoop Common
  1. Hadoop Common
  2. HADOOP-5884

Capacity scheduler should account high memory jobs as using more capacity of the queue

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Fixes Capacity scheduler to account more capacity of a queue for a high memory job. Done by considering these jobs to
      take more slots proportionally with respect to a slot's default memory size.
      Show
      Fixes Capacity scheduler to account more capacity of a queue for a high memory job. Done by considering these jobs to take more slots proportionally with respect to a slot's default memory size.

      Description

      Currently, when a high memory job is scheduled by the capacity scheduler, each task scheduled counts only once in the capacity of the queue, though it may actually be preventing other jobs from using spare slots on that node because of its higher memory requirements. In order to be fair, the capacity scheduler should proportionally (with respect to default memory) account high memory jobs as using a larger capacity of the queue.

      1. HADOOP-5884-20090605-branch-20.txt
        69 kB
        Vinod Kumar Vavilapalli
      2. HADOOP-5884.patch
        66 kB
        Hemanth Yamijala
      3. HADOOP-5884-20090603.txt
        69 kB
        Vinod Kumar Vavilapalli
      4. HADOOP-5884-20090602.1.txt
        64 kB
        Vinod Kumar Vavilapalli
      5. HADOOP-5884-20090529.1.txt
        45 kB
        Vinod Kumar Vavilapalli

        Issue Links

          Activity

            People

            • Assignee:
              Vinod Kumar Vavilapalli
              Reporter:
              Hemanth Yamijala
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development