[OOZIE-2716] Oozie Truncates Workflow Parameter Values which contain '}}'

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 4.1.0

Environment

Using Oozie 4.1.0-cdh5.8.2, Java 1.8.0_60-b27

• Assignee: Unassigned
• Reporter: Ruben Aguilar
• Created: 21/Oct/16 01:21
• Updated: 21/Oct/16 20:49
• Resolved: 21/Oct/16 20:49

Description

When submitting a workflow job to oozie through the web api, if the value of a substituted workflow parameter contains a ""}} then the rest of the value is truncated. This happens quite often which json data. I tried delimiting by

}} => }}}}
}} => \}\}
}} => '}''}'

but none of these worked.

Example:

Expected value received in jar (ending double braces):

{"hdfsArtifacts":null,"azureSqlArtifacts":null,"osdbArtifacts":{"username":"","password":"","server":"","database":"","schema":"","scriptsPath":"","outputType":""}}

Actual value received in jar (lacking ending double braces):

{"hdfsArtifacts":null,"azureSqlArtifacts":null,"osdbArtifacts":{"username":"","password":"","server":"","database":"","schema":"","scriptsPath":"","outputType":""

workflow.xml:

<workflow-app name="end-2-end-test" xmlns="uri:oozie:workflow:0.4">
  <start to="SparkAction" />
  <action name="SparkAction">
    <spark xmlns="uri:oozie:spark-action:0.1">
      <job-tracker>${jobTracker}</job-tracker>
      <name-node>${nameNode}</name-node>
      <master>${master}</master>
      <mode>cluster</mode>
      <name>${jobName}</name>
      <class>${classToExecute}</class>
      <jar>${nameNode}${executableJarPath}</jar>
      <spark-opts></spark-opts>
      <arg>${artifactSpecification}</arg>
    </spark>
    <ok to="real-end" />
    <error to="fail"/>
  </action>
  <kill name="fail">
    <message>Spark Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
  </kill>
  <end name="real-end" />
</workflow-app>

Configuration sent to web api:

<configuration>
  <property>
    <name>jobName</name>
    <value>end-2-end-test</value>
  </property>
  <property>
    <name>oozie.use.system.libpath</name>
    <value>True</value>
  </property>
  <property>
    <name>executableJarPath</name>
    <value>/oozie-jars/e2e-job-test/version-14/E2EJobTest-assembly-1.20.jar</value>
  </property>
  <property>
    <name>user.name</name>
    <value>oozie</value>
  </property>
  <property>
    <name>mapreduce.job.user.name</name>
    <value>oozie</value>
  </property>
  <property>
    <name>master</name>
    <value>yarn-cluster</value>
  </property>
  <property>
    <name>oozie.wf.application.path</name>
    <value>hdfs://some-hdfs:8020/oozie-workflow/e2e-job-test/version-14/workflow.xml</value>
  </property>
  <property>
    <name>artifactSpecification</name>
    <value>{"hdfsArtifacts":null,"azureSqlArtifacts":null,"osdbArtifacts":{"username":"","password":"","server":"","database":"","schema":"","scriptsPath":"","outputType":""}}</value>
  </property>
  <property>
    <name>jobTracker</name>
    <value>some-namenode:8032</value>
  </property>
  <property>
    <name>nameNode</name>
    <value>hdfs://some-namenode:8020</value>
  </property>
  <property>
    <name>classToExecute</name>
    <value>E2EJob</value>
  </property>
</configuration>

Issue Links

• duplicates: SPARK-17814 spark submit arguments are truncated in yarn-cluster mode