[YARN-5793] Trim configuration values in DockerLinuxContainerRuntime

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha1
• Fix Version/s: 2.9.0, 3.0.0-alpha2
• Component/s: nodemanager
• Labels: Docker
• Flags: Patch
• Assignee: Tianyin Xu
• Reporter: Tianyin Xu
• Created: 27/Oct/16 23:00
• Updated: 02/May/18 22:34
• Resolved: 31/Oct/16 21:02

Description

The current implementation of DockerLinuxContainerRuntime does not follow the practice of trimming configuration values. This leads to errors if users set values containing space or newline.

see the following YARN commits as reference:
YARN-3395. FairScheduler: Trim whitespaces when using username for queuename.
YARN-2869. CapacityScheduler should trim sub queue names when parse configuration.
YARN-2843. Fixed NodeLabelsManager to trim inputs for hosts and labels so as to make them work correctly.

and many other Hadoop/HDFS commits (just list a few):
HDFS-9708. FSNamesystem.initAuditLoggers() doesn't trim classnames
HDFS-2799. Trim fs.checkpoint.dir values.
HADOOP-6578. Configuration should trim whitespace around a lot of value types
HADOOP-6534. Trim whitespace from directory lists initializing

Patch is available against trunk

DockerLinuxContainerRuntime.java

@@ -219,9 +219,9 @@ public void initialize(Configuration conf)
     dockerClient = new DockerClient(conf);
     allowedNetworks.clear();
     allowedNetworks.addAll(Arrays.asList(
-        conf.getStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
+        conf.getTrimmedStrings(YarnConfiguration.NM_DOCKER_ALLOWED_CONTAINER_NETWORKS,
             YarnConfiguration.DEFAULT_NM_DOCKER_ALLOWED_CONTAINER_NETWORKS)));
-    defaultNetwork = conf.get(
+    defaultNetwork = conf.getTrimmed(
         YarnConfiguration.NM_DOCKER_DEFAULT_CONTAINER_NETWORK,
         YarnConfiguration.DEFAULT_NM_DOCKER_DEFAULT_CONTAINER_NETWORK);
 
@@ -237,7 +237,7 @@ public void initialize(Configuration conf)
       throw new ContainerExecutionException(message);
     }
 
-    privilegedContainersAcl = new AccessControlList(conf.get(
+    privilegedContainersAcl = new AccessControlList(conf.getTrimmed(
         YarnConfiguration.NM_DOCKER_PRIVILEGED_CONTAINERS_ACL,
         YarnConfiguration.DEFAULT_NM_DOCKER_PRIVILEGED_CONTAINERS_ACL));
   }
@@ -439,7 +439,7 @@ public void launchContainer(ContainerRuntimeContext ctx)
         LOCALIZED_RESOURCES);
     @SuppressWarnings("unchecked")
     List<String> userLocalDirs = ctx.getExecutionAttribute(USER_LOCAL_DIRS);
-    Set<String> capabilities = new HashSet<>(Arrays.asList(conf.getStrings(
+    Set<String> capabilities = new HashSet<>(Arrays.asList(conf.getTrimmedStrings(
         YarnConfiguration.NM_DOCKER_CONTAINER_CAPABILITIES,
         YarnConfiguration.DEFAULT_NM_DOCKER_CONTAINER_CAPABILITIES)));