[AMBARI-8028] After adding hdp.version in yarn-site.xml Distributed Shell is failing

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.7.0
• Fix Version/s: 1.7.0
• Component/s: ambari-server
• Assignee: Alejandro Fernandez
• Reporter: Alejandro Fernandez
• Created: 29/Oct/14 20:40
• Updated: 13/May/15 07:42
• Resolved: 29/Oct/14 22:27

Description

After adding hdp.version in yarn-site.xml Distributed Shell is failing

 /usr/hdp/current/hadoop-yarn-client/bin/yarn  org.apache.hadoop.yarn.applications.distributedshell.Client -jar /usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.jar -shell_command pwd

Failing with classpath issue:

Diagnostics: Exception from container-launch.
Container id: container_1414595622703_0039_02_000001
Exit code: 1
Exception message: /grid/1/yarn/local/usercache/hrt_qa/appcache/application_1414595622703_0039/container_1414595622703_0039_02_000001/launch_container.sh: line 9: $CLASSPATH:./*:$HADOOP_CONF_DIR:/usr/hdp/${hdp.version}/hadoop-client/*:/usr/hdp/${hdp.version}/hadoop-client/lib/*:/usr/hdp/${hdp.version}/hadoop-hdfs-client/*:/usr/hdp/${hdp.version}/hadoop-hdfs-client/lib/*:/usr/hdp/${hdp.version}/hadoop-yarn-client/*:/usr/hdp/${hdp.version}/hadoop-yarn-client/lib/*:./log4j.properties: bad substitution

Stack trace: ExitCodeException exitCode=1: /grid/1/yarn/local/usercache/hrt_qa/appcache/application_1414595622703_0039/container_1414595622703_0039_02_000001/launch_container.sh: line 9: $CLASSPATH:./*:$HADOOP_CONF_DIR:/usr/hdp/${hdp.version}/hadoop-client/*:/usr/hdp/${hdp.version}/hadoop-client/lib/*:/usr/hdp/${hdp.version}/hadoop-hdfs-client/*:/usr/hdp/${hdp.version}/hadoop-hdfs-client/lib/*:/usr/hdp/${hdp.version}/hadoop-yarn-client/*:/usr/hdp/${hdp.version}/hadoop-yarn-client/lib/*:./log4j.properties: bad substitution

        at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
        at org.apache.hadoop.util.Shell.run(Shell.java:455)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
        at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:206)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:301)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)


Container exited with a non-zero exit code 1

Though in secure this not reported as front-end error, AM logs show classpath issue there also

Container: container_1414601160300_0005_01_000001 on karam-tez-sec-exp1-3.cs1cloud.internal_45454
===================================================================================================
LogType: AppMaster.stderr
LogUploadTime: 29-Oct-2014 16:51:42
LogLength: 996
Log Contents:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/exceptions/YarnException
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2570)
        at java.lang.Class.getMethod0(Class.java:2813)
        at java.lang.Class.getMethod(Class.java:1663)
        at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.exceptions.YarnException
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

Issue Links

• links to: Code Review patch