Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5884

Capacity scheduler should account high memory jobs as using more capacity of the queue

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.20.1
    • None
    • None
    • Reviewed
    • Hide
      Fixes Capacity scheduler to account more capacity of a queue for a high memory job. Done by considering these jobs to
      take more slots proportionally with respect to a slot's default memory size.
      Show
      Fixes Capacity scheduler to account more capacity of a queue for a high memory job. Done by considering these jobs to take more slots proportionally with respect to a slot's default memory size.

    Description

      Currently, when a high memory job is scheduled by the capacity scheduler, each task scheduled counts only once in the capacity of the queue, though it may actually be preventing other jobs from using spare slots on that node because of its higher memory requirements. In order to be fair, the capacity scheduler should proportionally (with respect to default memory) account high memory jobs as using a larger capacity of the queue.

      Attachments

        1. HADOOP-5884-20090605-branch-20.txt
          69 kB
          Vinod Kumar Vavilapalli
        2. HADOOP-5884.patch
          66 kB
          Hemanth Yamijala
        3. HADOOP-5884-20090603.txt
          69 kB
          Vinod Kumar Vavilapalli
        4. HADOOP-5884-20090602.1.txt
          64 kB
          Vinod Kumar Vavilapalli
        5. HADOOP-5884-20090529.1.txt
          45 kB
          Vinod Kumar Vavilapalli

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            vinodkv Vinod Kumar Vavilapalli
            yhemanth Hemanth Yamijala
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment