[YARN-8643] Docker image life cycle management on HDFS

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.0
• Component/s: yarn
• Labels: Docker
• Target Version/s: 3.3.0
• Assignee: Eric Yang
• Reporter: Eric Yang
• Created: 09/Aug/18 17:19
• Updated: 08/Mar/19 16:49
• Resolved: 08/Mar/19 16:49

Description

This JIRA is forking an idea proposed in YARN-3854 to manage trusted docker images on HDFS. YARN-3854 is going to focus on rearranging the node manager and container-executor logic to download docker image during file localization phase for better error handling and reporting. The new proposal is to create a docker container which host Docker registry and NFS gateway, and configure docker images to store on HDFS. The docker container can run in YARN framework to serve as privileged or trusted registries.

HDFS_Docker_registry_design.png describes the components required to create a docker image that can store docker images on HDFS.

Deploy_HDFS_Docker_registry_workflow.png describes the operation to perform to setup a trusted docker registry that write to HDFS.

Issue Links

• is a clone of: YARN-9228 [Umbrella] Docker image life cycle management on HDFS