Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
Hadoop 2.X has the weird behavior to translate yarn.app.mapreduce.am.env,
JAVA_HOME=/home/gs/java/jdk64/current,JAVA_HOME=/home/gs/java/jdk64/current
to
JAVA_HOME=/home/gs/java/jdk64/current:JAVA_HOME=/home/gs/java/jdk64/current
due to this, some of env variables are not taken in effect properly.
this problem happens when running launcher with uber mode in Hadoop 2.X,
since oozie copies mapreduce.map.env to yarn.app.mapreduce.am.env, like following case
<property> <name>oozie.launcher.mapreduce.map.env</name> <value>JAVA_HOME=/home/gs/java/jdk64/current</value> </property> <property> <name>oozie.launcher.yarn.app.mapreduce.am.env</name> <value>JAVA_HOME=/home/gs/java/jdk64/current</value> </property>
this patch is to do
1) Deduplication before appending map.env to am.env
2) If there is a conflict, turn of uber mode AM.
Attachments
Attachments
Issue Links
- links to