[YARN-6988] container-executor fails for docker when command length > 4096 B

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.9.0, 3.0.0-beta1, 2.8.2
• Component/s: yarn
• Labels: Docker
• Hadoop Flags: Reviewed
• Assignee: Eric Badger
• Reporter: Eric Badger
• Created: 10/Aug/17 21:15
• Updated: 02/May/18 22:34
• Resolved: 17/Aug/17 20:55

Description

run_docker and launch_docker_container_as_user allocate their command arrays using EXECUTOR_PATH_MAX, which is hardcoded to 4096 in configuration.h. Because of this, the full docker command can only be 4096 characters. If it is longer, it will be truncated and the command will fail with a parsing error. Because of the bind-mounting of volumes, the arguments to the docker command can quickly get large. For example, I passed the 4096 limit with an 11 disk node.

Issue Links

• is duplicated by: YARN-6623 Add support to turn off launching privileged containers in the container-executor