Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9860

Enable service mode for Docker containers on YARN



    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.3.0
    • Fix Version/s: 3.3.0
    • Component/s: None
    • Labels:


      This task is to add support to YARN for running Docker containers in "Service Mode".

      Service Mode - Run the container as defined by the image, but still allow for injecting configuration.

      Entrypoint mode helped - now able to use the ENV and ENTRYPOINT/CMD as defined in the image. However, still requires modification to official images due to user propagation
      User propagation is problematic for running a secure cluster with sssd

      Must be enabled via c-e.cfg (example: docker.service-mode.allowed=true)
      Must be requested at runtime - (example: YARN_CONTAINER_RUNTIME_DOCKER_SERVICE_MODE=true)
      Entrypoint mode is default enabled for this mode (If Service Mode is requested, YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE should be set to true)
      Writable log mount will not be added - stdout logging may still work with entrypoint mode - remove the writable bind mounts
      User and groups will not be propagated (now: docker run --user nobody --group-add=nobody .... <image>, after: docker run .... <image>)
      Read-only resources mounted at the file level, files get chmod 777, parent directory only accessible by the run as user.

      cc Shane Kumpf


        1. YARN-9860-001.patch
          36 kB
          Prabhu Joseph
        2. YARN-9860-002.patch
          36 kB
          Prabhu Joseph
        3. YARN-9860-003.patch
          37 kB
          Prabhu Joseph
        4. YARN-9860-004.patch
          37 kB
          Prabhu Joseph
        5. YARN-9860-005.patch
          39 kB
          Prabhu Joseph
        6. YARN-9860-006.patch
          39 kB
          Prabhu Joseph
        7. YARN-9860-007.patch
          39 kB
          Prabhu Joseph
        8. Screen Shot 2019-10-09 at 11.27.19 AM.png
          214 kB
          Prabhu Joseph
        9. YARN-9860-008.patch
          41 kB
          Prabhu Joseph
        10. YARN-9860-009.patch
          43 kB
          Prabhu Joseph



            • Assignee:
              prabhujoseph Prabhu Joseph
              prabhujoseph Prabhu Joseph
            • Votes:
              0 Vote for this issue
              6 Start watching this issue


              • Created: