Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-12

Support for multiple job-xml elements in action nodes

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.3.0
    • None

    Description

      There is a use case when having ability to specify multiple job-xml elements in action node would help in reducing the complexity of Oozie workflow and simplify WF configuration maintenance.

      The basic idea is to split configuration properties into multiple files (per component for example) and re-use these files for different actions in the WF.
      For example, we can have following configuration files available:

      • app-db-config.xml
      • app-ws-config.xml
      • app-hbase-config.xml
      • app-hadoop-config.xml
      • ...

      Now, when action in WF requires properties from one or two configuration file, we would be able to specify only those files:

      <workflow-app name="foo-wf" xmlns="uri:oozie:workflow:0.1">
      ...
      <action name="MultipleJobXMLsAction">
      <map-reduce>
      <job-tracker>foo:9001</job-tracker>
      <name-node>bar:9000</name-node>
      <job-xml>/app-db-config.xml</job-xml>
      <job-xml>/app-hbase-config.xml</job-xml>
      <configuration>
      <property>
      <name>mapred.input.dir</name>
      <value>/usr/tucu/input-data</value>
      </property>
      <property>
      <name>mapred.output.dir</name>
      <value>/usr/tucu/input-data</value>
      </property>
      <property>
      <name>mapred.reduce.tasks</name>
      <value>${firstJobReducers}</value>
      </property>
      </configuration>
      </map-reduce>
      <ok to="myNextAction"/>
      <error to="errorCleanup"/>
      </action>
      ...
      </workflow-app>

      This functionality can be easily implemented inside setup method for Hadoop mapper class but it would be more elegant to have it exposed as part of Oozie WF

      Attachments

        1. 12-v4.patch
          36 kB
          Robert Kanter
        2. 12-v3.patch
          20 kB
          Robert Kanter
        3. 12-v2.patch
          18 kB
          Robert Kanter
        4. 12.patch
          12 kB
          Robert Kanter

        Issue Links

          Activity

            People

              rkanter Robert Kanter
              samir.eljazovic Samir Eljazovic
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: