Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4161

Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      Capacity Scheduler right now schedules multiple containers per heart beat if there are more resources available in the node.
      This approach works fine however in some cases its not distribute the load across the cluster hence throughput of the cluster suffers. I am adding feature to drive that using configuration by that we can control the number of containers assigned per heart beat.

      Attachments

        1. YARN-4161.patch.1
          16 kB
          Wei Yan
        2. YARN-4161.patch
          16 kB
          Mayank Bansal
        3. YARN-4161.branch-2.006.patch
          18 kB
          Wangda Tan
        4. YARN-4161.006.patch
          18 kB
          Wei Yan
        5. YARN-4161.005.patch
          18 kB
          Wei Yan
        6. YARN-4161.004.patch
          19 kB
          Wei Yan
        7. YARN-4161.003.patch
          17 kB
          Wei Yan
        8. YARN-4161.002.patch
          18 kB
          Wei Yan

        Issue Links

        is related to

        Improvement - An improvement or enhancement to an existing feature or task. YARN-6851 Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved
        relates to

        Improvement - An improvement or enhancement to an existing feature or task. YARN-10860 Make max container per heartbeat configs refreshable

        • Major - Major loss of function.
        • Resolved

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ywskycn Wei Yan Assign to me
            mayank_bansal Mayank Bansal
            Votes:
            2 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment