Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7512 Support service upgrade via YARN Service API and CLI
  3. YARN-8160

Yarn Service Upgrade: Support upgrade of service that use docker containers



    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.2.0, 3.1.2
    • None


      Ability to upgrade dockerized yarn native services.
      Ref: YARN-5637

      Container upgrade is supported by the NM via reInitializeContainer api. reInitializeContainer does NOT change the ContainerId of the upgraded container.
      NM performs the following steps during reInitializeContainer:

      • kills the existing process
      • cleans up the container
      • launches another container with the new ContainerLaunchContext

      NOTE: ContainerLaunchContext holds all the information that needs to upgrade the container.

      With reInitializeContainer, the following does NOT change

      • container ID. This is not created by NM. It is provided to it and here RM is not creating another container allocation.
      • localizedResources this stays the same if the upgrade does NOT require additional resources IIUC.

      The following changes with reInitializeContainer

      • the working directory of the upgraded container changes. It is NOT a relaunch.

      Changes required in the case of docker container

      • reInitializeContainer seems to not be working with Docker containers. Investigate and fix this.
      • [Future change] Add an additional api to NM to pull the images and modify reInitializeContainer to trigger docker container launch without pulling the image first which could be based on a flag.
        • When the service upgrade is initialized, we can provide the user with an option to just pull the images on the NMs.
        • When a component instance is upgrade, it calls the reInitializeContainer with the flag pull-image set to false, since the NM will have already pulled the images.


        1. container_e02_1533231998644_0009_01_000003.nm.log
          371 kB
          Chandni Singh
        2. YARN-8160.001.patch
          3 kB
          Chandni Singh
        3. YARN-8160.002.patch
          6 kB
          Chandni Singh
        4. YARN-8160.003.patch
          10 kB
          Chandni Singh
        5. YARN-8160.004.patch
          26 kB
          Chandni Singh
        6. YARN-8160-branch-3.1.002.patch
          26 kB
          Chandni Singh
        7. YARN-8160-branch-3.1.003.patch
          26 kB
          Chandni Singh



            csingh Chandni Singh
            csingh Chandni Singh
            0 Vote for this issue
            8 Start watching this issue