Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1197

Support changing resources of an allocated container

Add voteVotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      The current YARN resource management logic assumes resource allocated to a container is fixed during the lifetime of it. When users want to change a resource
      of an allocated container the only way is releasing it and allocating a new container with expected size.
      Allowing run-time changing resources of an allocated container will give us better control of resource usage in application side

        Attachments

        1. YARN-1197_Design.2015.06.24.pdf
          685 kB
          MENG DING
        2. YARN-1197_Design.2015.07.07.pdf
          700 kB
          MENG DING
        3. YARN-1197_Design.2015.08.21.pdf
          692 kB
          MENG DING
        4. YARN-1197_Design.pdf
          692 kB
          MENG DING
        5. YARN-1197 old-design-docs-patches-for-reference.zip
          813 kB
          Wangda Tan

        Issue Links

        1.
        Common PB type definitions for container resizing Sub-task Closed Wangda Tan Actions
        2.
        AM-RM protocol changes to support container resizing Sub-task Closed Wangda Tan Actions
        3.
        AM-RM protocol changes to support container resizing Sub-task Resolved MENG DING Actions
        4.
        AM-NM protocol changes to support container resizing Sub-task Resolved MENG DING Actions
        5.
        ContainerManager implementation to support container resizing Sub-task Resolved MENG DING Actions
        6.
        RM-NM protocol changes and NodeStatusUpdater implementation to support container resizing Sub-task Resolved MENG DING Actions
        7.
        ContainerImpl changes to support container resizing Sub-task Resolved MENG DING Actions
        8.
        ContainerManager recovery for container resizing Sub-task Resolved MENG DING Actions
        9.
        Make ContainersMonitor can support change monitoring size of an allocated container in NM side Sub-task Resolved MENG DING Actions
        10.
        CapacityScheduler side changes to support increase/decrease container resource. Sub-task Resolved Wangda Tan Actions
        11.
        Support change container resource in RM Sub-task Resolved Wangda Tan Actions
        12.
        Roll back container resource allocation after resource increase token expires Sub-task Resolved MENG DING Actions
        13.
        Make AMRMClient support send increase container request and get increased/decreased containers Sub-task Resolved MENG DING Actions
        14.
        Make NMClient support change container resources Sub-task Resolved MENG DING Actions
        15.
        Add implementations to FairScheduler to support increase/decrease container resource Sub-task Patch Available Wilfred Spiegelenburg Actions
        16.
        Protocol changes in RM side to support change container resource Sub-task Resolved Unassigned Actions
        17.
        [YARN-1197] Add increased/decreased container to Allocation Sub-task Resolved Unassigned Actions
        18.
        [YARN-1197] Modify ApplicationMasterService to support changing container resource Sub-task Resolved Unassigned Actions
        19.
        [YARN-1197] Modify ResourceTrackerService to support passing decreased containers to RMNode Sub-task Resolved Unassigned Actions
        20.
        [YARN-1197] Add pullDecreasedContainer API to RMNode which can be used by scheduler to get newly decreased Containers Sub-task Resolved Unassigned Actions
        21.
        [YARN-1197] Add methods in FiCaSchedulerNode to support increase/decrease/reserve/unreserve change container requests/results Sub-task Resolved Unassigned Actions
        22.
        [YARN-1197] Add APIs in CSQueue to support decrease container resource and unreserve increase request Sub-task Resolved Unassigned Actions
        23.
        [YARN-1197] Add implementations to CapacityScheduler to support increase/decrease container resource Sub-task Resolved Unassigned Actions
        24.
        Merge YARN-1197 back to trunk Sub-task Resolved Wangda Tan Actions
        25.
        Support changing container cpu resource Sub-task Resolved Yang Wang Actions
        26.
        Resolve findbugs/javac warnings in YARN-1197 branch Sub-task Resolved Wangda Tan Actions
        27.
        Example of use YARN-1197 Sub-task In Progress MENG DING Actions
        28.
        Increasing container resource while there is no headroom left will cause ResourceManager to crash Sub-task Resolved MENG DING Actions

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              leftnoteasy Wangda Tan

              Dates

              • Created:
                Updated:

                Issue deployment