[MAPREDUCE-6322] MapReduce Integration tests with mini cluster fail on Jenkins when path to workspace contains characters which need to be escaped in shell scripts

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.4.0

Environment

Jenkins on Linux

• Assignee: Unassigned
• Reporter: Jens Rabe
• Created: 20/Apr/15 10:31
• Updated: 20/Apr/15 10:31

Description

When building a project with Jenkins which contains integration tests for Map/Reduce jobs, the tests fail when the path to the workspace Jenkins builds in contains characters which need to be escaped in shell scripts, like whitespaces or parentheses.

Example output:

/application_1429525168682_0001/container_1429525168682_0001_02_000001/default_container_executor.sh: line 3: `echo $$ > /home/jenkins/data/workspace/Some project with whitespaces (and parentheses)/path/to/com.example.someproject/target/test-cluster-cff13b33-ab3f-443d-9e50-c28ce37e9a2a/test-cluster-cff13b33-ab3f-443d-9e50-c28ce37e9a2a-localDir-nm-1_0/nmPrivate/container_1429525168682_0001_02_000001.pid.tmp'

	at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
	at org.apache.hadoop.util.Shell.run(Shell.java:418)
	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
	at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)


Container exited with a non-zero exit code 2
.Failing this attempt.. Failing the application.

The script default_container_executor.sh should be generated in a way that such characters are escaped.