[YARN-1298] Unable to remotely submit job from Windows client to Linux server

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.1.0-beta
• Component/s: client

Environment

Windows 7 client, Ubuntu server

• Assignee: Unassigned
• Reporter: Mircea Gaceanu
• Created: 11/Oct/13 08:29
• Updated: 18/Mar/14 18:50
• Resolved: 18/Mar/14 18:50

Description

When submitting a job from a Windows client, the following error occurs:

2013-10-10 15:36:52,936 INFO  [main]: Job                        - Job job_1381404247909_0007 failed with state FAILED due to: Application application_1381404247909_0007 failed 2 times due to AM Container for appattempt_1381404247909_0007_000002 exited with  exitCode: 1 due to: Exception from container-launch: 
org.apache.hadoop.util.Shell$ExitCodeException: /bin/bash: line 0: fg: no job control

	at org.apache.hadoop.util.Shell.runCommand(Shell.java:458)
	at org.apache.hadoop.util.Shell.run(Shell.java:373)
	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:578)
	at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:258)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:74)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
	at java.lang.Thread.run(Thread.java:662)


.Failing this attempt.. Failing the application.

This happens because of the Shell.WINDOWS constant is TRUE, so when the YARNRunner starts the ApplicationManager, the JAVA_HOME env var evaluates to windows-style:

2013-10-10 15:36:48,798 DEBUG [main]: YARNRunner                 - Command to launch container for ApplicationMaster is : %JAVA_HOME%/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA  -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1><LOG_DIR>/stdout 2><LOG_DIR>/stderr

Issue Links

• is duplicated by: MAPREDUCE-4052 Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
• is duplicated by: YARN-1824 Make Windows client work with Linux/Unix cluster