[OOZIE-1486] Cut down on number of small files created to track a running action

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: trunk
• Fix Version/s: 4.1.0
• Labels: performance scalability
• Assignee: Mona Chitnis
• Reporter: Mona Chitnis
• Created: 03/Aug/13 01:21
• Updated: 09/Dec/14 00:45
• Resolved: 28/Aug/13 18:49

Description

Oozie creates multiple files while running a action. This has been observed to be an overkill and can be consolidated (as applicable) into a lesser files. Advantages involve not only staying within user storage quotas but also reducing Namenode pressure in a large production environment.

static final String ACTION_CONF_XML = "action.xml";
public static final String ACTION_PREPARE_XML = "oozie.action.prepare.xml";
private static final String ACTION_OUTPUT_PROPS = "output.properties";
private static final String ACTION_STATS_PROPS = "stats.properties";
private static final String ACTION_EXTERNAL_CHILD_IDS_PROPS =
"externalChildIds.properties";
private static final String ACTION_NEW_ID_PROPS = "newId.properties";
private static final String ACTION_ERROR_PROPS = "error.properties";

Consolidate and reduce the number of files required.

Issue Links

• is required by: OOZIE-1487 Oozie SSH actions creates staging directories in local fil-system and never cleans up
• relates to: OOZIE-1520 Sequencefile Reader fails to use doas for reading action data file
• links to: ReviewBoard

Sub-Tasks

• LauncherMapper to check for sys properties before opening files for action data: Closed
• Maintain backward incompatibility for running jobs before upgrade: Closed