Description
There is a use case when having ability to specify multiple job-xml elements in action node would help in reducing the complexity of Oozie workflow and simplify WF configuration maintenance.
The basic idea is to split configuration properties into multiple files (per component for example) and re-use these files for different actions in the WF.
For example, we can have following configuration files available:
- app-db-config.xml
- app-ws-config.xml
- app-hbase-config.xml
- app-hadoop-config.xml
- ...
Now, when action in WF requires properties from one or two configuration file, we would be able to specify only those files:
<workflow-app name="foo-wf" xmlns="uri:oozie:workflow:0.1">
...
<action name="MultipleJobXMLsAction">
<map-reduce>
<job-tracker>foo:9001</job-tracker>
<name-node>bar:9000</name-node>
<job-xml>/app-db-config.xml</job-xml>
<job-xml>/app-hbase-config.xml</job-xml>
<configuration>
<property>
<name>mapred.input.dir</name>
<value>/usr/tucu/input-data</value>
</property>
<property>
<name>mapred.output.dir</name>
<value>/usr/tucu/input-data</value>
</property>
<property>
<name>mapred.reduce.tasks</name>
<value>${firstJobReducers}</value>
</property>
</configuration>
</map-reduce>
<ok to="myNextAction"/>
<error to="errorCleanup"/>
</action>
...
</workflow-app>
This functionality can be easily implemented inside setup method for Hadoop mapper class but it would be more elegant to have it exposed as part of Oozie WF
Attachments
Attachments
Issue Links
- is duplicated by
-
OOZIE-869 Multiple job-xml elements per action
- Resolved