Hadoop YARN
  1. Hadoop YARN
  2. YARN-1197

Support changing resources of an allocated container

    Details

      Description

      The current YARN resource management logic assumes resource allocated to a container is fixed during the lifetime of it. When users want to change a resource
      of an allocated container the only way is releasing it and allocating a new container with expected size.
      Allowing run-time changing resources of an allocated container will give us better control of resource usage in application side

      1. YARN-1197_Design.2015.08.21.pdf
        692 kB
        MENG DING
      2. YARN-1197_Design.2015.07.07.pdf
        700 kB
        MENG DING
      3. YARN-1197_Design.2015.06.24.pdf
        685 kB
        MENG DING
      4. YARN-1197 old-design-docs-patches-for-reference.zip
        813 kB
        Wangda Tan
      5. YARN-1197_Design.pdf
        692 kB
        MENG DING

        Issue Links

        1.
        Common PB type definitions for container resizing Sub-task Closed Wangda Tan
         
        2.
        AM-RM protocol changes to support container resizing Sub-task Closed Wangda Tan
         
        3.
        AM-RM protocol changes to support container resizing Sub-task Resolved MENG DING
         
        4.
        AM-NM protocol changes to support container resizing Sub-task Resolved MENG DING
         
        5.
        ContainerManager implementation to support container resizing Sub-task Resolved MENG DING
         
        6.
        RM-NM protocol changes and NodeStatusUpdater implementation to support container resizing Sub-task Resolved MENG DING
         
        7.
        ContainerImpl changes to support container resizing Sub-task Resolved MENG DING
         
        8.
        ContainerManager recovery for container resizing Sub-task Resolved MENG DING
         
        9.
        Make ContainersMonitor can support change monitoring size of an allocated container in NM side Sub-task Resolved MENG DING
         
        10.
        CapacityScheduler side changes to support increase/decrease container resource. Sub-task Resolved Wangda Tan
         
        11.
        Support change container resource in RM Sub-task Resolved Wangda Tan
         
        12. Roll back container resource allocation after resource increase token expires Sub-task Patch Available MENG DING
         
        13.
        Make AMRMClient support send increase container request and get increased/decreased containers Sub-task Resolved MENG DING
         
        14.
        Make NMClient support change container resources Sub-task Resolved MENG DING
         
        15. Add implementations to FairScheduler to support increase/decrease container resource Sub-task Open Sandy Ryza
         
        16.
        Protocol changes in RM side to support change container resource Sub-task Resolved Unassigned
         
        17.
        [YARN-1197] Add increased/decreased container to Allocation Sub-task Resolved Unassigned
         
        18.
        [YARN-1197] Modify ApplicationMasterService to support changing container resource Sub-task Resolved Unassigned
         
        19.
        [YARN-1197] Modify ResourceTrackerService to support passing decreased containers to RMNode Sub-task Resolved Unassigned
         
        20.
        [YARN-1197] Add pullDecreasedContainer API to RMNode which can be used by scheduler to get newly decreased Containers Sub-task Resolved Unassigned
         
        21.
        [YARN-1197] Add methods in FiCaSchedulerNode to support increase/decrease/reserve/unreserve change container requests/results Sub-task Resolved Unassigned
         
        22.
        [YARN-1197] Add APIs in CSQueue to support decrease container resource and unreserve increase request Sub-task Resolved Unassigned
         
        23.
        [YARN-1197] Add implementations to CapacityScheduler to support increase/decrease container resource Sub-task Resolved Unassigned
         
        24.
        Merge YARN-1197 back to trunk Sub-task Resolved Wangda Tan
         
        25. Support changing container cpu resource Sub-task Open Naganarasimha G R
         
        26.
        Resolve findbugs/javac warnings in YARN-1197 branch Sub-task Resolved Wangda Tan
         
        27. Example of use YARN-1197 Sub-task In Progress MENG DING
         
        28.
        Increasing container resource while there is no headroom left will cause ResourceManager to crash Sub-task Resolved MENG DING
         

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Wangda Tan
            • Votes:
              14 Vote for this issue
              Watchers:
              98 Start watching this issue

              Dates

              • Created:
                Updated:

                Development