Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2818

Can't overwrite oozie.action.max.output.data on a per-workflow basis

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0.0b1
    • Component/s: None
    • Labels:
      None

      Description

      It's not possible to raise the limit of the output on a per-action basis.
      With the workflow:

      <workflow-app name="LargeOutputWorkflow" xmlns="uri:oozie:workflow:0.5">
      <start to="shell-e207"/>
      <kill name="Kill">
      <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
      </kill>
      <action name="shell-e207">
      <shell xmlns="uri:oozie:shell-action:0.1">
      <job-tracker>${jobTracker}</job-tracker>
      <name-node>${nameNode}</name-node>
      <configuration>
      <property>
      <name>oozie.action.max.output.data</name>
      <value>100000</value>
      </property>
      <property>
      <name>oozie.launcher.oozie.action.max.output.data</name>
      <value>100000</value>
      </property>
      </configuration>
      <exec>/user/admin/large_output.sh</exec>
      <argument>10</argument>
      <file>/user/admin/large_output.sh#large_output.sh</file>
      <capture-output/>
      </shell>
      <ok to="End"/>
      <error to="Kill"/>
      </action>
      <end name="End"/>
      </workflow-app>

      In the localized files in the job.xml the value of this property gets overwritten:

      1. cat /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_000002/job.xml |grep max.outp
        <property><name>oozie.launcher.oozie.action.max.output.data</name><value>100000</value><source>programatically</source><source>job.xml</source></property>
        <property><name>oozie.action.max.output.data</name><value>2048</value><source>programatically</source><source>job.xml</source></property>
      2. cat /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_000002/action.xml |grep max.outp
        <property><name>oozie.launcher.oozie.action.max.output.data</name><value>100000</value><source>programatically</source></property>
        <property><name>oozie.action.max.output.data</name><value>100000</value><source>programatically</source></property>

      I could not find a reason for this behavior, so I think it's a bug.

      The value is read from ConfigurationService and set in createLauncherConf in JavaActionExecutor. There could be a check to see if it's already set.

        Attachments

        1. OOZIE-2818-01.patch
          3 kB
          Attila Sasvári

          Activity

            People

            • Assignee:
              asasvari Attila Sasvári
              Reporter:
              gezapeti Gézapeti
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: