Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5884

Capacity scheduler should account high memory jobs as using more capacity of the queue

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Fixes Capacity scheduler to account more capacity of a queue for a high memory job. Done by considering these jobs to
      take more slots proportionally with respect to a slot's default memory size.
      Show
      Fixes Capacity scheduler to account more capacity of a queue for a high memory job. Done by considering these jobs to take more slots proportionally with respect to a slot's default memory size.

      Description

      Currently, when a high memory job is scheduled by the capacity scheduler, each task scheduled counts only once in the capacity of the queue, though it may actually be preventing other jobs from using spare slots on that node because of its higher memory requirements. In order to be fair, the capacity scheduler should proportionally (with respect to default memory) account high memory jobs as using a larger capacity of the queue.

        Attachments

        1. HADOOP-5884-20090605-branch-20.txt
          69 kB
          Vinod Kumar Vavilapalli
        2. HADOOP-5884.patch
          66 kB
          Hemanth Yamijala
        3. HADOOP-5884-20090603.txt
          69 kB
          Vinod Kumar Vavilapalli
        4. HADOOP-5884-20090602.1.txt
          64 kB
          Vinod Kumar Vavilapalli
        5. HADOOP-5884-20090529.1.txt
          45 kB
          Vinod Kumar Vavilapalli

          Issue Links

            Activity

              People

              • Assignee:
                vinodkv Vinod Kumar Vavilapalli
                Reporter:
                yhemanth Hemanth Yamijala
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: