Oozie
  1. Oozie
  2. OOZIE-693

Fork-join validation doesn't check for the 'error to' transition of nodes

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2.0
    • Component/s: None
    • Labels:
      None

      Description

      This is in addition to Oozie-636. Fork-join validation doesn't check for the 'error to' transition of nodes.

        Activity

        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/3883/
        -----------------------------------------------------------

        Review request for oozie and Santhosh Srinivasan.

        Summary
        -------

        https://issues.apache.org/jira/browse/OOZIE-693

        This addresses bug OOZIE-693.
        https://issues.apache.org/jira/browse/OOZIE-693

        Diffs


        trunk/core/src/main/java/org/apache/oozie/ErrorCode.java 1243647
        trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 1243647
        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java 1243647

        Diff: https://reviews.apache.org/r/3883/diff

        Testing
        -------

        Few tests added

        Thanks,

        Virag

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3883/ ----------------------------------------------------------- Review request for oozie and Santhosh Srinivasan. Summary ------- https://issues.apache.org/jira/browse/OOZIE-693 This addresses bug OOZIE-693 . https://issues.apache.org/jira/browse/OOZIE-693 Diffs trunk/core/src/main/java/org/apache/oozie/ErrorCode.java 1243647 trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 1243647 trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java 1243647 Diff: https://reviews.apache.org/r/3883/diff Testing ------- Few tests added Thanks, Virag
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/3883/#review5064
        -----------------------------------------------------------

        A few comments.

        trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11110>

        Extra line

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11111>

        Extra line

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11103>

        Change the method to testTransitionFailure1()

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11104>

        Change the method to testTransitionFailure2()

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11105>

        Include a message for the failure.

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11106>

        Change the method to testTransitionFailure3()

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11107>

        Include a message for the failure.

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11109>

        Can a fork have multiple incoming edges?

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11108>

        Include a message for the failure.

        • Santhosh

        On 2012-02-13 19:10:15, Virag Kothari wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/3883/

        -----------------------------------------------------------

        (Updated 2012-02-13 19:10:15)

        Review request for oozie and Santhosh Srinivasan.

        Summary

        -------

        https://issues.apache.org/jira/browse/OOZIE-693

        This addresses bug OOZIE-693.

        https://issues.apache.org/jira/browse/OOZIE-693

        Diffs

        -----

        trunk/core/src/main/java/org/apache/oozie/ErrorCode.java 1243647

        trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 1243647

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java 1243647

        Diff: https://reviews.apache.org/r/3883/diff

        Testing

        -------

        Few tests added

        Thanks,

        Virag

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3883/#review5064 ----------------------------------------------------------- A few comments. trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11110 > Extra line trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11111 > Extra line trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11103 > Change the method to testTransitionFailure1() trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11104 > Change the method to testTransitionFailure2() trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11105 > Include a message for the failure. trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11106 > Change the method to testTransitionFailure3() trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11107 > Include a message for the failure. trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11109 > Can a fork have multiple incoming edges? trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11108 > Include a message for the failure. Santhosh On 2012-02-13 19:10:15, Virag Kothari wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3883/ ----------------------------------------------------------- (Updated 2012-02-13 19:10:15) Review request for oozie and Santhosh Srinivasan. Summary ------- https://issues.apache.org/jira/browse/OOZIE-693 This addresses bug OOZIE-693 . https://issues.apache.org/jira/browse/OOZIE-693 Diffs ----- trunk/core/src/main/java/org/apache/oozie/ErrorCode.java 1243647 trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 1243647 trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java 1243647 Diff: https://reviews.apache.org/r/3883/diff Testing ------- Few tests added Thanks, Virag
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/3883/#review5065
        -----------------------------------------------------------

        trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11112>

        k

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11113>

        k

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11114>

        k

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11115>

        k

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11116>

        k

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11117>

        k

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11119>

        Yes.. We are more liberal in error-checking for 'error-to' transition of nodes.
        E.g. Forked actions can directly transition to kill node on 'error-to'. No need of a join there.

        Also, fork can have multiple edges from a decision node.

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
        <https://reviews.apache.org/r/3883/#comment11118>

        k

        • Virag

        On 2012-02-13 19:10:15, Virag Kothari wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/3883/

        -----------------------------------------------------------

        (Updated 2012-02-13 19:10:15)

        Review request for oozie and Santhosh Srinivasan.

        Summary

        -------

        https://issues.apache.org/jira/browse/OOZIE-693

        This addresses bug OOZIE-693.

        https://issues.apache.org/jira/browse/OOZIE-693

        Diffs

        -----

        trunk/core/src/main/java/org/apache/oozie/ErrorCode.java 1243647

        trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 1243647

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java 1243647

        Diff: https://reviews.apache.org/r/3883/diff

        Testing

        -------

        Few tests added

        Thanks,

        Virag

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3883/#review5065 ----------------------------------------------------------- trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11112 > k trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11113 > k trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11114 > k trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11115 > k trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11116 > k trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11117 > k trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11119 > Yes.. We are more liberal in error-checking for 'error-to' transition of nodes. E.g. Forked actions can directly transition to kill node on 'error-to'. No need of a join there. Also, fork can have multiple edges from a decision node. trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java < https://reviews.apache.org/r/3883/#comment11118 > k Virag On 2012-02-13 19:10:15, Virag Kothari wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3883/ ----------------------------------------------------------- (Updated 2012-02-13 19:10:15) Review request for oozie and Santhosh Srinivasan. Summary ------- https://issues.apache.org/jira/browse/OOZIE-693 This addresses bug OOZIE-693 . https://issues.apache.org/jira/browse/OOZIE-693 Diffs ----- trunk/core/src/main/java/org/apache/oozie/ErrorCode.java 1243647 trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 1243647 trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java 1243647 Diff: https://reviews.apache.org/r/3883/diff Testing ------- Few tests added Thanks, Virag
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/3883/
        -----------------------------------------------------------

        (Updated 2012-02-14 00:20:21.166003)

        Review request for oozie and Santhosh Srinivasan.

        Changes
        -------

        Addressing Santhosh's comments

        Summary
        -------

        https://issues.apache.org/jira/browse/OOZIE-693

        This addresses bug OOZIE-693.
        https://issues.apache.org/jira/browse/OOZIE-693

        Diffs (updated)


        trunk/core/src/main/java/org/apache/oozie/ErrorCode.java 1243751
        trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 1243751
        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java 1243751

        Diff: https://reviews.apache.org/r/3883/diff

        Testing
        -------

        Few tests added

        Thanks,

        Virag

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3883/ ----------------------------------------------------------- (Updated 2012-02-14 00:20:21.166003) Review request for oozie and Santhosh Srinivasan. Changes ------- Addressing Santhosh's comments Summary ------- https://issues.apache.org/jira/browse/OOZIE-693 This addresses bug OOZIE-693 . https://issues.apache.org/jira/browse/OOZIE-693 Diffs (updated) trunk/core/src/main/java/org/apache/oozie/ErrorCode.java 1243751 trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 1243751 trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java 1243751 Diff: https://reviews.apache.org/r/3883/diff Testing ------- Few tests added Thanks, Virag
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/3883/#review5079
        -----------------------------------------------------------

        Ship it!

        +1

        • Santhosh

        On 2012-02-14 00:20:21, Virag Kothari wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/3883/

        -----------------------------------------------------------

        (Updated 2012-02-14 00:20:21)

        Review request for oozie and Santhosh Srinivasan.

        Summary

        -------

        https://issues.apache.org/jira/browse/OOZIE-693

        This addresses bug OOZIE-693.

        https://issues.apache.org/jira/browse/OOZIE-693

        Diffs

        -----

        trunk/core/src/main/java/org/apache/oozie/ErrorCode.java 1243751

        trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 1243751

        trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java 1243751

        Diff: https://reviews.apache.org/r/3883/diff

        Testing

        -------

        Few tests added

        Thanks,

        Virag

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3883/#review5079 ----------------------------------------------------------- Ship it! +1 Santhosh On 2012-02-14 00:20:21, Virag Kothari wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3883/ ----------------------------------------------------------- (Updated 2012-02-14 00:20:21) Review request for oozie and Santhosh Srinivasan. Summary ------- https://issues.apache.org/jira/browse/OOZIE-693 This addresses bug OOZIE-693 . https://issues.apache.org/jira/browse/OOZIE-693 Diffs ----- trunk/core/src/main/java/org/apache/oozie/ErrorCode.java 1243751 trunk/core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 1243751 trunk/core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java 1243751 Diff: https://reviews.apache.org/r/3883/diff Testing ------- Few tests added Thanks, Virag
        Hide
        Virag Kothari added a comment -

        For the patch committed in trunk with this JIRA ID (also available under the same JIRA ID in Apache ReviewBoard system), I grant license to ASF for inclusion in ASF works (as per the Apache License §5)

        Show
        Virag Kothari added a comment - For the patch committed in trunk with this JIRA ID (also available under the same JIRA ID in Apache ReviewBoard system), I grant license to ASF for inclusion in ASF works (as per the Apache License §5)
        Hide
        Nathaniel Troutman added a comment -

        Not sure how to reopen this, but the following workflow now fails. It simple deletes the output forks the creation of two directories and finishes. It fails with the error:

            Error: E0735 : E0735: There was an invalid "error to" transition to node [send-error-email] while using fork/join
        

        Which I have reduced down to each node using the same "error to" transition to send an error email before moving to done.

        <workflow-app xmlns="uri:oozie:workflow:0.2" name="simple">
        	<start to="prep"/>
        	
        	<!-- this node will remove everything before stuff gets started  -->
        	<action name="prep">
                <fs><delete path='${rootOutpath}'/></fs>
                <ok to="fork-it"/>
                <error to="send-error-email"/>
            </action>
            
         
         	<!-- create two directories -->
        	<fork name="fork-it">
        		<path start="path1"/>
        		<path start="path2"/>
        	</fork>
        	
        		<action name="path1">
        			<fs><mkdir path='${rootOutpath}/path1'/></fs>
        			<ok to="join-it"/>
        			<error to="send-error-email"/>
        		</action>
        		
        		<action name="path2">
        			<fs><mkdir path='${rootOutpath}/path2'/></fs>
        			<ok to="join-it"/>
        			<error to="send-error-email"/>
        		</action>
        				
        	<join name="join-it" to="send-done-email"/>
        	
        	<action name="send-done-email">
        			<email xmlns="uri:oozie:email-action:0.1">
        				<to>${wf:user()}@mail.com</to>
        				<subject>Oozie Workflow Complete ${wf:name()} on ${clusterName}</subject>
        				<body>Done</body>
        			</email>
        			<ok to="done"/>
        			<error to="kill"/>		 
        	</action>
        	
        	
        	<action name="send-error-email">
        		<email xmlns="uri:oozie:email-action:0.1">
        			<to>${wf:user()}@mail.com</to>
        			<subject>Oozie Workflow Failure ${wf:name()} action ${wf:lastErrorNode()} on ${clusterName}</subject>
        			<body>Error Message: ${wf:errorMessage(wf:lastErrorNode())}</body>
        		</email>
        		<ok to="done"/>
        		<error to="kill"/>		 
        	</action>
        	
        	<kill name="kill">
        		<message>FATAL: unable to send email on action failure.</message>
        	</kill>
        	
        	<end name="done"/>
        </workflow-app>
        
        Show
        Nathaniel Troutman added a comment - Not sure how to reopen this, but the following workflow now fails. It simple deletes the output forks the creation of two directories and finishes. It fails with the error: Error: E0735 : E0735: There was an invalid "error to" transition to node [send-error-email] while using fork/join Which I have reduced down to each node using the same "error to" transition to send an error email before moving to done. <workflow-app xmlns= "uri:oozie:workflow:0.2" name= "simple" > <start to= "prep" /> <!-- this node will remove everything before stuff gets started --> <action name= "prep" > <fs> <delete path='${rootOutpath}'/> </fs> <ok to= "fork-it" /> <error to= "send-error-email" /> </action> <!-- create two directories --> <fork name= "fork-it" > <path start= "path1" /> <path start= "path2" /> </fork> <action name= "path1" > <fs> <mkdir path='${rootOutpath}/path1'/> </fs> <ok to= "join-it" /> <error to= "send-error-email" /> </action> <action name= "path2" > <fs> <mkdir path='${rootOutpath}/path2'/> </fs> <ok to= "join-it" /> <error to= "send-error-email" /> </action> <join name= "join-it" to= "send-done-email" /> <action name= "send-done-email" > <email xmlns= "uri:oozie:email-action:0.1" > <to> ${wf:user()}@mail.com </to> <subject> Oozie Workflow Complete ${wf:name()} on ${clusterName} </subject> <body> Done </body> </email> <ok to= "done" /> <error to= "kill" /> </action> <action name= "send-error-email" > <email xmlns= "uri:oozie:email-action:0.1" > <to> ${wf:user()}@mail.com </to> <subject> Oozie Workflow Failure ${wf:name()} action ${wf:lastErrorNode()} on ${clusterName} </subject> <body> Error Message: ${wf:errorMessage(wf:lastErrorNode())} </body> </email> <ok to= "done" /> <error to= "kill" /> </action> <kill name= "kill" > <message> FATAL: unable to send email on action failure. </message> </kill> <end name= "done" /> </workflow-app>
        Hide
        Virag Kothari added a comment -

        This is recently addressed in OOZIE-1035

        Show
        Virag Kothari added a comment - This is recently addressed in OOZIE-1035

          People

          • Assignee:
            Virag Kothari
            Reporter:
            Virag Kothari
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development