Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8472

YARN Container Phase 2

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Hide
      - Improved debugging Docker container on YARN
      - Improved security for running Docker containers
      - Improved cgroup management for docker container.
      Show
      - Improved debugging Docker container on YARN - Improved security for running Docker containers - Improved cgroup management for docker container.

      Description

      In YARN-3611, we have implemented basic Docker container support for YARN.  This story is the next phase to improve container usability.

      Several area for improvements are:

      1. Software defined network support
      2. Interactive shell to container
      3. User management sss/nscd integration
      4. Runc/containerd support
      5. Metrics/Logs integration with Timeline service v2
      6. Docker container profiles
      7. Docker cgroup management

        Attachments

        Issue Links

        1.
        check docker container's exit code when writing to cgroup task files Sub-task Resolved Jim Brennan   Actions
        2.
        Allow administrators to control available container runtimes and set defaults for all containers Sub-task Resolved Craig Condit   Actions
        3.
        Force removal of docker containers that do not get removed on first try Sub-task Resolved Eric Yang   Actions
        4.
        Update documentation and yarn-default related to the Docker runtime Sub-task Resolved Craig Condit   Actions
        5.
        DockerClient still touches hadoop.tmp.dir Sub-task Resolved Craig Condit   Actions
        6.
        Separate white list for docker.trusted.registries and docker.privileged-container.registries Sub-task Resolved Eric Yang   Actions
        7.
        Privileged, trusted containers should be supported only in ENTRYPOINT mode Sub-task Resolved Eric Yang   Actions
        8.
        Add port mapping handling when docker container use bridge network Sub-task Resolved Xun Liu   Actions
        9.
        Document best practice for user management Sub-task Resolved Eric Yang   Actions
        10.
        Interactive docker shell Sub-task Resolved Zian Chen   Actions
        11.
        Create an interface to provide cluster information to application Sub-task Resolved Eric Yang   Actions
        12.
        Automate docker network configuration through YARN API Sub-task Resolved Unassigned   Actions
        13.
        Update Docker examples to use image which exists Sub-task Resolved Craig Condit   Actions
        14.
        Cleanup ENTRYPOINT documentation Sub-task Resolved Craig Condit   Actions
        15.
        Add support for tmpfs mounts with the Docker runtime Sub-task Resolved Craig Condit   Actions
        16.
        Docker image life cycle management on HDFS Sub-task Resolved Eric Yang   Actions
        17.
        Expose container's hostname to applications running within the docker container Sub-task Resolved Suma Shivaprasad   Actions
        18.
        Add support for network-alias in docker run for user defined networks Sub-task Resolved Suma Shivaprasad   Actions
        19.
        NM gets backed up deleting docker containers Sub-task Resolved Chandni Singh   Actions
        20.
        DelayedProcessKiller is executed for Docker containers even though docker stop sends a KILL signal after the specified grace period Sub-task Resolved Chandni Singh   Actions
        21.
        Improve the error message when a bind mount is not whitelisted Sub-task Resolved Simon Prewo

        0%

        Original Estimate - 2h
        Remaining Estimate - 2h
        Actions
        22.
        Improve the documentation for the docker.trusted.registries configuration Sub-task Resolved Eric Yang   Actions
        23.
        When STOPSIGNAL is not present then docker inspect returns an extra line feed Sub-task Resolved Chandni Singh   Actions
        24.
        In some cases docker kill is used to stop non-privileged containers instead of sending the signal directly Sub-task Resolved Chandni Singh   Actions
        25.
        javadoc comments in docker-util.h is confusing Sub-task Resolved Zian Chen   Actions
        26.
        Automatically convert the launch command to the exec form when using entrypoint support Sub-task Resolved Eric Yang   Actions
        27.
        Add a flag to use local docker image instead of getting latest from registry Sub-task Resolved Chandni Singh   Actions
        28.
        Support multi-homed network for docker container Sub-task Resolved Unassigned   Actions
        29.
        publish all exposed ports to random ports when using bridge network Sub-task Resolved dockerzhang   Actions
        30.
        Docker container rm command should be executed after stop Sub-task Resolved Zhaohui Xin   Actions
        31.
        Carriage Return character in launch command cause node manager to become unhealthy Sub-task Resolved Billie Rinaldi   Actions
        32.
        Container reinit always fails in branch-3.2 and trunk Sub-task Resolved Chandni Singh   Actions
        33.
        Docker run doesn't pull down latest image if the image exists locally Sub-task Resolved Zhaohui Xin   Actions
        34.
        Implement logic to keep docker image consistent in application that uses :latest tag Sub-task Resolved Eric Yang   Actions
        35.
        Disable PATH variable to be passed to Docker container Sub-task Resolved Jim Brennan   Actions
        36.
        Docker container exited with failure does not get clean up correctly Sub-task Resolved Eric Yang   Actions
        37.
        Reinit of container stuck in localization Sub-task Resolved Eric Yang   Actions
        38.
        Enhance documentation of Docker on YARN support Sub-task Resolved Peter Bacsko   Actions
        39.
        Remove reapDockerContainerNoPid left behind by YARN-9074 Sub-task Resolved Kevin Su   Actions

          Activity

            People

            • Assignee:
              eyang Eric Yang
              Reporter:
              eyang Eric Yang

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 2h
                2h
                Remaining:
                Remaining Estimate - 2h
                2h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Issue deployment