[OOZIE-2818] Can't overwrite oozie.action.max.output.data on a per-workflow basis

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 5.0.0b1
• Assignee: Attila Sasvári
• Reporter: Gézapeti
• Created: 10/Mar/17 11:57
• Updated: 25/Jan/18 21:00
• Resolved: 19/Apr/17 11:43

Description

It's not possible to raise the limit of the output on a per-action basis.
With the workflow:

<workflow-app name="LargeOutputWorkflow" xmlns="uri:oozie:workflow:0.5">
<start to="shell-e207"/>
<kill name="Kill">
<message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<action name="shell-e207">
<shell xmlns="uri:oozie:shell-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>oozie.action.max.output.data</name>
<value>100000</value>
</property>
<property>
<name>oozie.launcher.oozie.action.max.output.data</name>
<value>100000</value>
</property>
</configuration>
<exec>/user/admin/large_output.sh</exec>
<argument>10</argument>
<file>/user/admin/large_output.sh#large_output.sh</file>
<capture-output/>
</shell>
<ok to="End"/>
<error to="Kill"/>
</action>
<end name="End"/>
</workflow-app>

In the localized files in the job.xml the value of this property gets overwritten:

1. cat /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_000002/job.xml |grep max.outp
   <property><name>oozie.launcher.oozie.action.max.output.data</name><value>100000</value><source>programatically</source><source>job.xml</source></property>
   <property><name>oozie.action.max.output.data</name><value>2048</value><source>programatically</source><source>job.xml</source></property>
2. cat /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_000002/action.xml |grep max.outp
   <property><name>oozie.launcher.oozie.action.max.output.data</name><value>100000</value><source>programatically</source></property>
   <property><name>oozie.action.max.output.data</name><value>100000</value><source>programatically</source></property>

I could not find a reason for this behavior, so I think it's a bug.

The value is read from ConfigurationService and set in createLauncherConf in JavaActionExecutor. There could be a check to see if it's already set.