Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3920

FairScheduler: Limit node reservations to large containers

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • fairscheduler
    • None

    Description

      Reserving a node for a container was designed for preventing large containers from starvation from small requests that keep getting into a node. Today we let this be used even for a small container request. This has a huge impact on scheduling since we block other scheduling requests until that reservation is fulfilled. We should make this configurable so its impact can be minimized by limiting it for large container requests as originally intended.

      Attachments

        1. yARN-3920.001.patch
          12 kB
          Anubhav Dhoot
        2. yARN-3920.002.patch
          12 kB
          Anubhav Dhoot
        3. yARN-3920.003.patch
          16 kB
          Anubhav Dhoot
        4. YARN-3920.004.patch
          13 kB
          Anubhav Dhoot
        5. YARN-3920.004.patch
          13 kB
          Anubhav Dhoot
        6. YARN-3920.004.patch
          13 kB
          Anubhav Dhoot
        7. YARN-3920.004.patch
          15 kB
          Anubhav Dhoot
        8. YARN-3920.005.patch
          14 kB
          Anubhav Dhoot

        Issue Links

        is related to

        Improvement - An improvement or enhancement to an existing feature or task. YARN-7356 The fair scheduler reservation threshold should be documented in the fair scheduler docs

        • Major - Major loss of function.
        • Open

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            adhoot Anubhav Dhoot Assign to me
            adhoot Anubhav Dhoot
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment