[YARN-8762] [Umbrella] Support Interactive Docker Shell to running Containers

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.0
• Labels: Docker
• Target Version/s: 3.3.0
• Release Note: - Add shell access to YARN containers
• Assignee: Eric Yang
• Reporter: Zian Chen
• Created: 10/Sep/18 23:35
• Updated: 20/Dec/18 01:20
• Resolved: 20/Dec/18 01:20

Description

Debugging distributed application can be challenging on Hadoop. Hadoop provide limited debugging ability through application log files. One of the most frequently requested feature is to provide interactive shell to assist real time debugging. This feature is inspired by docker exec to provide ability to run arbitrary commands in docker container.

Issue Links

• is related to: YARN-8523 Interactive docker shell

Sub-Tasks

• Add WebSocket logic to the Node Manager web server to establish servlet: Resolved
• Container Executor change to create stdin/stdout pipeline: Resolved
• Container Executor C binary change to execute interactive docker command: Resolved
• Add Command Line interface to invoke interactive docker shell: Resolved
• Authentication Filter change to force security check: Resolved
• Add security check for container user is same as websocket user: Resolved
• Define a protocol exchange between websocket client and server for interactive shell: Resolved
• Add xtermjs to YARN UI2: Resolved
• Add ability to use interactive shell with normal yarn container: Resolved
• Add flag to disable interactive shell: Resolved
• Support sh shell for interactive container shell at command line: Resolved
• Web browser close without proper exit can leak shell process: Resolved
• Add Terminal Link to Service component instance page for UI2: Resolved
• Improve terminal message when connection is refused: Resolved
• Container shell does not work when using yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user is set: Resolved
• Ensure flush after printing to log plus additional cleanup: Resolved