[CRUNCH-217] Crunch PipelineResult objects do not capture failures

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6.0
• Fix Version/s: 0.7.0
• Component/s: Core
• Assignee: Josh Wills
• Reporter: Joseph Adler
• Created: 10/Jun/13 15:56
• Updated: 29/Jul/13 15:51
• Resolved: 10/Jun/13 16:44

Description

We use Crunch within Azkaban (a scheduling a and workflow tool, similar to Oozie). We need to be able to run a crunch pipeline from within another Java program, then test for success using code like this:

    PipelineResult mainResult = mainPipeline.run();
    if (!mainResult.succeeded())
    {
        throw new Exception("Pipeline failed: " + mainPipeline.getName() + "\n");
    }
    return 0;

Currently, PipelineResult objects don't let you do this; the succeeded() method will always return true.

I have enclosed a patch that addresses this issue for in-memory and map-reduce pipelines.