Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9860

Enable service mode for Docker containers on YARN

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.3.0
    • Fix Version/s: 3.3.0
    • Component/s: None
    • Labels:
      None

      Description

      This task is to add support to YARN for running Docker containers in "Service Mode".

      Service Mode - Run the container as defined by the image, but still allow for injecting configuration.

      Background:
      Entrypoint mode helped - now able to use the ENV and ENTRYPOINT/CMD as defined in the image. However, still requires modification to official images due to user propagation
      User propagation is problematic for running a secure cluster with sssd

      Implementation:
      Must be enabled via c-e.cfg (example: docker.service-mode.allowed=true)
      Must be requested at runtime - (example: YARN_CONTAINER_RUNTIME_DOCKER_SERVICE_MODE=true)
      Entrypoint mode is default enabled for this mode (If Service Mode is requested, YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE should be set to true)
      Writable log mount will not be added - stdout logging may still work with entrypoint mode - remove the writable bind mounts
      User and groups will not be propagated (now: docker run --user nobody --group-add=nobody .... <image>, after: docker run .... <image>)
      Read-only resources mounted at the file level, files get chmod 777, parent directory only accessible by the run as user.

      cc Shane Kumpf

        Attachments

        1. YARN-9860-001.patch
          36 kB
          Prabhu Joseph
        2. YARN-9860-002.patch
          36 kB
          Prabhu Joseph
        3. YARN-9860-003.patch
          37 kB
          Prabhu Joseph
        4. YARN-9860-004.patch
          37 kB
          Prabhu Joseph
        5. YARN-9860-005.patch
          39 kB
          Prabhu Joseph
        6. YARN-9860-006.patch
          39 kB
          Prabhu Joseph
        7. YARN-9860-007.patch
          39 kB
          Prabhu Joseph
        8. Screen Shot 2019-10-09 at 11.27.19 AM.png
          214 kB
          Prabhu Joseph
        9. YARN-9860-008.patch
          41 kB
          Prabhu Joseph
        10. YARN-9860-009.patch
          43 kB
          Prabhu Joseph

          Activity

            People

            • Assignee:
              prabhujoseph Prabhu Joseph
              Reporter:
              prabhujoseph Prabhu Joseph
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: