[OOZIE-2561] Child job url in case of child job failure

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Satish Saley
• Reporter: Satish Saley
• Created: 06/Jun/16 16:28
• Updated: 06/Jul/16 23:17
• Resolved: 06/Jul/16 23:17

Description

Currently we are putting child job ids in output.properties in

protected static void writeExternalChildIDs(String logFile, Pattern[] patterns, String name) {
        // Harvesting and recording Hadoop Job IDs
        // See JavaActionExecutor#readExternalChildIDs for how they are read
        try {
            Properties jobIds = getHadoopJobIds(logFile, patterns);
            File file = new File(System.getProperty(LauncherMapper.ACTION_PREFIX
                    + LauncherMapper.ACTION_DATA_OUTPUT_PROPS));
            OutputStream os = new FileOutputStream(file);
            try {
                jobIds.store(os, "");
            }
            finally {
                os.close();
            }
            System.out.println(" Hadoop Job IDs executed by " + name + ": " + jobIds.getProperty(HADOOP_JOBS));
            System.out.println();
        }
        catch (Exception e) {
            System.out.println("WARN: Error getting Hadoop Job IDs executed by " + name);
            e.printStackTrace(System.out);
        }
    }

We read from this file only in case of success of Launcher Main

In JavaActionExecutor:
if (LauncherMapperHelper.isMainSuccessful(runningJob)) {
                        if (getCaptureOutput(action) && LauncherMapperHelper.hasOutputData(actionData)) {
                            context.setExecutionData(SUCCEEDED, PropertiesUtils.stringToProperties(actionData
                                    .get(LauncherMapper.ACTION_DATA_OUTPUT_PROPS)));
                            LOG.info(XLog.STD, "action produced output");
                        }

We should write child job its to oozie.action.externalChildIDs. We need changes in following

• HiveMain
• Hive2Main
• DistcpMain
• SparkMain
• SqoopMain

Issue Links

• duplicates: OOZIE-2582 Populating external child Ids for action failures