Hadoop YARN
  1. Hadoop YARN
  2. YARN-1197

Support changing resources of an allocated container

    Details

    • Type: Task Task
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.1.0-beta
    • Fix Version/s: None
    • Labels:
      None

      Description

      The current YARN resource management logic assumes resource allocated to a container is fixed during the lifetime of it. When users want to change a resource
      of an allocated container the only way is releasing it and allocating a new container with expected size.
      Allowing run-time changing resources of an allocated container will give us better control of resource usage in application side

      1. yarn-server-resourcemanager.patch.ver.1
        115 kB
        Wangda Tan (No longer used)
      2. yarn-server-nodemanager.patch.ver.1
        33 kB
        Wangda Tan (No longer used)
      3. yarn-server-common.patch.ver.1
        10 kB
        Wangda Tan (No longer used)
      4. yarn-pb-impl.patch.ver.1
        70 kB
        Wangda Tan (No longer used)
      5. yarn-api-protocol.patch.ver.1
        18 kB
        Wangda Tan (No longer used)
      6. yarn-1197-v5.pdf
        162 kB
        Wangda Tan (No longer used)
      7. yarn-1197-v4.pdf
        162 kB
        Wangda Tan (No longer used)
      8. yarn-1197-v3.pdf
        162 kB
        Wangda Tan (No longer used)
      9. yarn-1197-v2.pdf
        162 kB
        Wangda Tan (No longer used)
      10. yarn-1197-scheduler-v1.pdf
        137 kB
        Wangda Tan (No longer used)
      11. yarn-1197.pdf
        132 kB
        Wangda Tan (No longer used)
      12. tools-project.patch.ver.1
        2 kB
        Wangda Tan (No longer used)
      13. mapreduce-project.patch.ver.1
        6 kB
        Wangda Tan (No longer used)

        Issue Links

        1.
        Common PB type definitions for container resizing Sub-task Closed Wangda Tan (No longer used)
         
        2.
        AM-RM protocol changes to support container resizing Sub-task Closed Wangda Tan (No longer used)
         
        3. Protocol changes and implementations in NM side to support change container resource Sub-task Open Wangda Tan
         
        4. Make AMRMClient support send increase container request and get increased/decreased containers Sub-task Open Wangda Tan (No longer used)
         
        5. Protocol changes and implementations in RM side to support change container resource Sub-task Open Wangda Tan (No longer used)
         
        6. Make NMClient support change container resources Sub-task Open Wangda Tan (No longer used)
         
        7. [YARN-1197] Make ContainersMonitor can support change monitoring size of an allocated container in NM side Sub-task Open Wangda Tan (No longer used)
         
        8. [YARN-1197] Add newly decreased container to NodeStatus in NM side Sub-task Open Wangda Tan (No longer used)
         
        9. [YARN-1197] Add changeContainersResource interface and implementations to ContainerManagementProtocol Sub-task Open Wangda Tan (No longer used)
         
        10. [YARN-1197] Add increase container request to YarnScheduler allocate API Sub-task Open Wangda Tan (No longer used)
         
        11. [YARN-1197] Add increased/decreased container to Allocation Sub-task Open Wangda Tan (No longer used)
         
        12. [YARN-1197] Modify ApplicationMasterService to support changing container resource Sub-task Open Wangda Tan (No longer used)
         
        13. [YARN-1197] Modify ResourceTrackerService to support passing decreased containers to RMNode Sub-task Open Wangda Tan (No longer used)
         
        14. [YARN-1197] Add pullDecreasedContainer API to RMNode which can be used by scheduler to get newly decreased Containers Sub-task Open Wangda Tan (No longer used)
         
        15. [YARN-1197] Add methods in FiCaSchedulerApp to support add/reserve/unreserve/allocate/pull change container requests/results Sub-task Open Wangda Tan (No longer used)
         
        16. [YARN-1197] Add methods in FiCaSchedulerNode to support increase/decrease/reserve/unreserve change container requests/results Sub-task Open Wangda Tan (No longer used)
         
        17. [YARN-1197] Add APIs in CSQueue to support decrease container resource and unreserve increase request Sub-task Open Wangda Tan (No longer used)
         
        18. [YARN-1197] Add implementations to CapacityScheduler to support increase/decrease container resource Sub-task Open Wangda Tan (No longer used)
         
        19. [YARN-1197] Add implementations to FairScheduler to support increase/decrease container resource Sub-task Open Sandy Ryza
         

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Wangda Tan
            • Votes:
              7 Vote for this issue
              Watchers:
              64 Start watching this issue

              Dates

              • Created:
                Updated:

                Development