[OOZIE-1958] address duplication of env variables in oozie.launcher.yarn.app.mapreduce.am.env when running with uber mode

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.1.0
• Assignee: Ryota Egashira
• Reporter: Ryota Egashira
• Created: 31/Jul/14 22:56
• Updated: 12/Jan/15 06:08
• Resolved: 11/Aug/14 19:44

Description

Hadoop 2.X has the weird behavior to translate yarn.app.mapreduce.am.env,
JAVA_HOME=/home/gs/java/jdk64/current,JAVA_HOME=/home/gs/java/jdk64/current
to
JAVA_HOME=/home/gs/java/jdk64/current:JAVA_HOME=/home/gs/java/jdk64/current

due to this, some of env variables are not taken in effect properly.

this problem happens when running launcher with uber mode in Hadoop 2.X,
since oozie copies mapreduce.map.env to yarn.app.mapreduce.am.env, like following case

<property>
      <name>oozie.launcher.mapreduce.map.env</name>
      <value>JAVA_HOME=/home/gs/java/jdk64/current</value>
</property>
<property>
      <name>oozie.launcher.yarn.app.mapreduce.am.env</name>
      <value>JAVA_HOME=/home/gs/java/jdk64/current</value>
</property>

this patch is to do
1) Deduplication before appending map.env to am.env
2) If there is a conflict, turn of uber mode AM.

Issue Links

• links to: RB