Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5884

Capacity scheduler should account high memory jobs as using more capacity of the queue

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.20.1
    • None
    • None
    • Reviewed
    • Hide
      Fixes Capacity scheduler to account more capacity of a queue for a high memory job. Done by considering these jobs to
      take more slots proportionally with respect to a slot's default memory size.
      Show
      Fixes Capacity scheduler to account more capacity of a queue for a high memory job. Done by considering these jobs to take more slots proportionally with respect to a slot's default memory size.

    Description

      Currently, when a high memory job is scheduled by the capacity scheduler, each task scheduled counts only once in the capacity of the queue, though it may actually be preventing other jobs from using spare slots on that node because of its higher memory requirements. In order to be fair, the capacity scheduler should proportionally (with respect to default memory) account high memory jobs as using a larger capacity of the queue.

      Attachments

        1. HADOOP-5884.patch
          66 kB
          Hemanth Yamijala
        2. HADOOP-5884-20090529.1.txt
          45 kB
          Vinod Kumar Vavilapalli
        3. HADOOP-5884-20090602.1.txt
          64 kB
          Vinod Kumar Vavilapalli
        4. HADOOP-5884-20090603.txt
          69 kB
          Vinod Kumar Vavilapalli
        5. HADOOP-5884-20090605-branch-20.txt
          69 kB
          Vinod Kumar Vavilapalli

        Issue Links

          Activity

            People

              vinodkv Vinod Kumar Vavilapalli
              yhemanth Hemanth Yamijala
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: