Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9860

Enable service mode for Docker containers on YARN

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.3.0
    • 3.3.0
    • None
    • None

    Description

      This task is to add support to YARN for running Docker containers in "Service Mode".

      Service Mode - Run the container as defined by the image, but still allow for injecting configuration.

      Background:
      Entrypoint mode helped - now able to use the ENV and ENTRYPOINT/CMD as defined in the image. However, still requires modification to official images due to user propagation
      User propagation is problematic for running a secure cluster with sssd

      Implementation:
      Must be enabled via c-e.cfg (example: docker.service-mode.allowed=true)
      Must be requested at runtime - (example: YARN_CONTAINER_RUNTIME_DOCKER_SERVICE_MODE=true)
      Entrypoint mode is default enabled for this mode (If Service Mode is requested, YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE should be set to true)
      Writable log mount will not be added - stdout logging may still work with entrypoint mode - remove the writable bind mounts
      User and groups will not be propagated (now: docker run --user nobody --group-add=nobody .... <image>, after: docker run .... <image>)
      Read-only resources mounted at the file level, files get chmod 777, parent directory only accessible by the run as user.

      cc shanekumpf@gmail.com

      Attachments

        1. YARN-9860-009.patch
          43 kB
          Prabhu Joseph
        2. YARN-9860-008.patch
          41 kB
          Prabhu Joseph
        3. Screen Shot 2019-10-09 at 11.27.19 AM.png
          214 kB
          Prabhu Joseph
        4. YARN-9860-007.patch
          39 kB
          Prabhu Joseph
        5. YARN-9860-006.patch
          39 kB
          Prabhu Joseph
        6. YARN-9860-005.patch
          39 kB
          Prabhu Joseph
        7. YARN-9860-004.patch
          37 kB
          Prabhu Joseph
        8. YARN-9860-003.patch
          37 kB
          Prabhu Joseph
        9. YARN-9860-002.patch
          36 kB
          Prabhu Joseph
        10. YARN-9860-001.patch
          36 kB
          Prabhu Joseph

        Activity

          People

            prabhujoseph Prabhu Joseph
            prabhujoseph Prabhu Joseph
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: