Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7469

Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      Queue Configuration:

      • Total Memory: 20GB
      • 2 Queues
        • Queue1
          • Memory: 10GB
          • MULP: 10%
          • ULF: 2.0
      • Minimum Container Size: 0.5GB

      Use Case:

      • User1 submits app1 to Queue1 and consumes 20GB
      • User2 submits app2 to Queue1 and requests 7.5GB
      • Preemption monitor preempts 7.5GB from app1. Capacity Scheduler gives those resources to User2
      • User 3 submits app3 to Queue1. To begin with, app3 is requesting 1 container for the AM.
      • Preemption monitor never preempts a container.

      Attachments

        1. YARN-7469.001.patch
          5 kB
          Eric Payne
        2. UnitTestToShowStarvedUser.patch
          2 kB
          Eric Payne

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            epayne Eric Payne
            epayne Eric Payne
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment