Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8472

YARN Container Phase 2

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      In YARN-3611, we have implemented basic Docker container support for YARN.  This story is the next phase to improve container usability.

      Several area for improvements are:

      1. Software defined network support
      2. Interactive shell to container
      3. User management sss/nscd integration
      4. Runc/containerd support
      5. Metrics/Logs integration with Timeline service v2
      6. Docker container profiles
      7. Docker cgroup management

        Attachments

          Issue Links

          1.
          check docker container's exit code when writing to cgroup task files Sub-task Resolved Jim Brennan  
          2.
          Allow administrators to control available container runtimes and set defaults for all containers Sub-task Resolved Craig Condit  
          3.
          Force removal of docker containers that do not get removed on first try Sub-task Resolved Eric Yang  
          4.
          Update documentation and yarn-default related to the Docker runtime Sub-task Resolved Craig Condit  
          5.
          DockerClient still touches hadoop.tmp.dir Sub-task Resolved Craig Condit  
          6.
          Separate white list for docker.trusted.registries and docker.privileged-container.registries Sub-task Resolved Eric Yang  
          7.
          Privileged, trusted containers should be supported only in ENTRYPOINT mode Sub-task Resolved Eric Yang  
          8.
          Add port mapping handling when docker container use bridge network Sub-task Resolved Xun Liu  
          9.
          Document best practice for user management Sub-task Resolved Eric Yang  
          10.
          Interactive docker shell Sub-task Resolved Zian Chen  
          11.
          Create an interface to provide cluster information to application Sub-task Resolved Eric Yang  
          12.
          Automate docker network configuration through YARN API Sub-task Resolved Unassigned  
          13.
          Update Docker examples to use image which exists Sub-task Resolved Craig Condit  
          14.
          Cleanup ENTRYPOINT documentation Sub-task Resolved Craig Condit  
          15.
          Add support for tmpfs mounts with the Docker runtime Sub-task Resolved Craig Condit  
          16.
          Docker image life cycle management on HDFS Sub-task Resolved Eric Yang  
          17.
          Expose container's hostname to applications running within the docker container Sub-task Resolved Suma Shivaprasad  
          18.
          Add support for network-alias in docker run for user defined networks Sub-task Open Suma Shivaprasad  
          19.
          NM gets backed up deleting docker containers Sub-task Resolved Chandni Singh  
          20.
          DelayedProcessKiller is executed for Docker containers even though docker stop sends a KILL signal after the specified grace period Sub-task Resolved Chandni Singh  
          21.
          Improve the error message when a bind mount is not whitelisted Sub-task Resolved Simon Prewo

          0%

          Original Estimate - 2h
          Remaining Estimate - 2h
          22.
          Improve the documentation for the docker.trusted.registries configuration Sub-task Resolved Eric Yang  
          23.
          When STOPSIGNAL is not present then docker inspect returns an extra line feed Sub-task Resolved Chandni Singh  
          24.
          In some cases docker kill is used to stop non-privileged containers instead of sending the signal directly Sub-task Open Chandni Singh  
          25.
          javadoc comments in docker-util.h is confusing Sub-task Resolved Zian Chen  
          26.
          Automatically convert the launch command to the exec form when using entrypoint support Sub-task Resolved Eric Yang  
          27.
          Add a flag to use local docker image instead of getting latest from registry Sub-task Resolved Chandni Singh  
          28.
          Support multi-homed network for docker container Sub-task Resolved Unassigned  
          29.
          publish all exposed ports to random ports when using bridge network Sub-task Resolved dockerzhang  
          30.
          Docker container rm command should be executed after stop Sub-task Resolved Zhaohui Xin  
          31.
          Carriage Return character in launch command cause node manager to become unhealthy Sub-task Resolved Billie Rinaldi  
          32.
          Container reinit always fails in branch-3.2 and trunk Sub-task Resolved Chandni Singh  
          33.
          Get the IP and port of the docker container and display it on WEB UI2 Sub-task Open Xun Liu  
          34.
          Docker run doesn't pull down latest image if the image exists locally Sub-task Resolved Zhaohui Xin  
          35.
          Implement logic to keep docker image consistent in application that uses :latest tag Sub-task Patch Available Eric Yang  
          36.
          Disable PATH variable to be passed to Docker container Sub-task Resolved Jim Brennan  
          37.
          Docker container exited with failure does not get clean up correctly Sub-task Resolved Eric Yang  
          38.
          Reinit of container stuck in localization Sub-task Resolved Eric Yang  
          39.
          Enhance documentation of Docker on YARN support Sub-task Resolved Peter Bacsko  
          40.
          Remove reapDockerContainerNoPid left behind by YARN-9074 Sub-task Resolved kevin su  

            Activity

              People

              • Assignee:
                eyang Eric Yang
                Reporter:
                eyang Eric Yang
              • Votes:
                0 Vote for this issue
                Watchers:
                28 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - 2h
                  2h
                  Remaining:
                  Remaining Estimate - 2h
                  2h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified