Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4333

Fair scheduler should support preemption within a leaf queue

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 2.6.0
    • None
    • fairscheduler
    • None

    Description

      Now each app in fair scheduler is allocated its fairshare, however fairshare resource is not ensured even if fairSharePreemption is enabled.
      Consider:
      1, When the cluster is idle, we submit app1 to queueA,which takes maxResource of queueA.
      2, Then the cluster becomes busy, but app1 does not release any resource, queueA resource usage is over its fairshare
      3, Then we submit app2(maybe with higher priority) to queueA. Now app2 has its own fairshare, but could not obtain any resource, since queueA is still over its fairshare and resource will not assign to queueA anymore. Also, preemption is not triggered in this case.
      So we should allow preemption within queue, when app is starved for fairshare.

      Attachments

        1. YARN-4333.001.patch
          13 kB
          Tao Jie
        2. YARN-4333.002.patch
          26 kB
          Tao Jie
        3. YARN-4333.003.patch
          28 kB
          Tao Jie

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Tao Jie Tao Jie
            Tao Jie Tao Jie
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment