Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-1130

Upgrade from 3.2 to 3.3 failing due to change in WorkflowInstance structure

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.3.0
    • Fix Version/s: 3.3.1
    • Component/s: None
    • Labels:
      None

      Description

      In 3.2, NodeHandlers were an inner class of NodeDefs. But after OOZIE-243, they are separate classes.
      So existing jobs of 3.2 fail on rerun, kill, suspend, resume after upgrading Oozie to 3.3

      Following is the exception trace

      $ oozie job -kill 0000032-121210224300123-oozie-oozi-W
      Error: E0607 : E0607: Other error in operation [java.lang.RuntimeException: java.io.IOException: java.lang.ClassNotFoundException: org.apache.oozie.workflow.lite.StartNodeDef$StartNodeHandler], {1}
       
       
       
       
      012-12-12 20:28:24,652 ERROR KillXCommand:536 - USER[mchiang] GROUP[users] TOKEN[-] APP[java-sleep] JOB[0000032-121210224300123-oozie-oozi-W] ACTION[-] Exception,
      java.lang.RuntimeException: java.io.IOException: java.lang.ClassNotFoundException: org.apache.oozie.workflow.lite.StartNodeDef$StartNodeHandler
              at org.apache.oozie.util.WritableUtils.fromByteArray(WritableUtils.java:70)
              at org.apache.oozie.WorkflowJobBean.get(WorkflowJobBean.java:356)
              at org.apache.oozie.WorkflowJobBean.getWorkflowInstance(WorkflowJobBean.java:219)
              at org.apache.oozie.command.wf.KillXCommand.execute(KillXCommand.java:119)
              at org.apache.oozie.command.wf.KillXCommand.execute(KillXCommand.java:54)
             at org.apache.oozie.command.XCommand.call(XCommand.java:277)
              at org.apache.oozie.DagEngine.kill(DagEngine.java:227)
              at org.apache.oozie.servlet.V1JobServlet.killWorkflowJob(V1JobServlet.java:465)
              at org.apache.oozie.servlet.V1JobServlet.killJob(V1JobServlet.java:140)
              at org.apache.oozie.servlet.BaseJobServlet.doPut(BaseJobServlet.java:95)
              at javax.servlet.http.HttpServlet.service(HttpServlet.java:640)
              at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:286)
              at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at com.yahoo.oozie.security.authentication.filter.YAuthFilter$2.doFilter(YAuthFilter.java:135)
              at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:372)
              at com.yahoo.oozie.security.authentication.filter.YAuthFilter.doFilter(YAuthFilter.java:139)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at yjava.servlet.FilterChainInvoker$ServletFilterChainInvoker.invoke(FilterChainInvoker.java:49)
              at yjava.servlet.filter.YHdrsFilter.doFilter(YHdrsFilter.java:68)
              at yjava.servlet.filter.YHdrsFilter.doFilter(YHdrsFilter.java:52)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at yjava.servlet.FilterChainInvoker$ServletFilterChainInvoker.invoke(FilterChainInvoker.java:49)
              at yjava.cookie.CookieDataFilter.doFilter(CookieDataFilter.java:447)
              at yjava.cookie.CookieDataFilter.doFilter(CookieDataFilter.java:219)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at yjava.servlet.FilterChainInvoker$ServletFilterChainInvoker.invoke(FilterChainInvoker.java:49)
              at yjava.servlet.filter.DoNotTrackFilter.doFilter(DoNotTrackFilter.java:104)
              at yjava.servlet.filter.DoNotTrackFilter.doFilter(DoNotTrackFilter.java:50)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at yjava.servlet.FilterChainInvoker$ServletFilterChainInvoker.invoke(FilterChainInvoker.java:49)
              at yjava.remote.ip.RemoteIPFilter.doFilter(RemoteIPFilter.java:104)
              at yjava.remote.ip.RemoteIPFilter.doFilter(RemoteIPFilter.java:65)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at yjava.servlet.FilterChainInvoker$ServletFilterChainInvoker.invoke(FilterChainInvoker.java:49)
              at yjava.security.yiv.servlet.InputValidationFilter.doFilter(InputValidationFilter.java:219)
              at yjava.security.yiv.servlet.InputValidationFilter.doFilter(InputValidationFilter.java:143)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
              at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
              at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
              at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
              at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
              at yjava.tomcat.valves.YahooConnectionValve.invoke(YahooConnectionValve.java:191)
              at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
              at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
              at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
              at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
              at java.lang.Thread.run(Thread.java:662)
      Caused by: java.io.IOException: java.lang.ClassNotFoundException: org.apache.oozie.workflow.lite.StartNodeDef$StartNodeHandler
              at org.apache.oozie.workflow.lite.NodeDef.readVersionOne(NodeDef.java:162)
              at org.apache.oozie.workflow.lite.NodeDef.readFields(NodeDef.java:188)
              at org.apache.oozie.workflow.lite.LiteWorkflowApp.readFields(LiteWorkflowApp.java:172)
              at org.apache.oozie.workflow.lite.LiteWorkflowInstance.readFields(LiteWorkflowInstance.java:593)
              at org.apache.oozie.util.WritableUtils.fromByteArray(WritableUtils.java:66)
              ... 55 more
      Caused by: java.lang.ClassNotFoundException: org.apache.oozie.workflow.lite.StartNodeDef$StartNodeHandler
              at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1680)
              at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1526)
              at java.lang.Class.forName0(Native Method)
              at java.lang.Class.forName(Class.java:169)
              at org.apache.oozie.workflow.lite.NodeDef.readVersionOne(NodeDef.java:159)
              ... 59 more
      
      
      1. OOZIE-1130.patch
        8 kB
        Virag Kothari
      2. OOZIE-1130.patch
        10 kB
        Virag Kothari

        Activity

        Hide
        virag Virag Kothari added a comment -

        Closing issue as it has been released as part of 3.3.1 release

        Show
        virag Virag Kothari added a comment - Closing issue as it has been released as part of 3.3.1 release
        Hide
        virag Virag Kothari added a comment -

        Committed to trunk and 3.3 after removing false change in license-headers. Thanks Alejandro and Rohini for review

        Show
        virag Virag Kothari added a comment - Committed to trunk and 3.3 after removing false change in license-headers. Thanks Alejandro and Rohini for review
        Hide
        hadoopqa Hadoop QA added a comment -

        Testing JIRA OOZIE-1130

        Cleaning local svn workspace

        ----------------------------

        +1 PATCH_APPLIES
        +1 CLEAN
        -1 RAW_PATCH_ANALYSIS
        . +1 the patch does not introduce any @author tags
        . +1 the patch does not introduce any tabs
        . +1 the patch does not introduce any trailing spaces
        . +1 the patch does not introduce any line longer than 132
        . -1 the patch does not add/modify any testcase
        +1 RAT
        . +1 the patch does not seem to introduce new RAT warnings
        +1 JAVADOC
        . +1 the patch does not seem to introduce new Javadoc warnings
        +1 COMPILE
        . +1 HEAD compiles
        . +1 patch compiles
        . +1 the patch does not seem to introduce new javac warnings
        +1 BACKWARDS_COMPATIBILITY
        . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
        . +1 the patch does not modify JPA files
        +1 TESTS
        . Tests run: 932
        +1 DISTRO
        . +1 distro tarball builds with the patch

        ----------------------------
        -1 Overall result, please check the reported -1(s)

        The full output of the test-patch run is available at

        . https://builds.apache.org/job/oozie-trunk-precommit-build/254/

        Show
        hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-1130 Cleaning local svn workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . -1 the patch does not add/modify any testcase +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files +1 TESTS . Tests run: 932 +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/254/
        Hide
        tucu00 Alejandro Abdelnur added a comment -

        Good catch Virag, my bad here. To avoid this in the future, we should create an integration test that has a few WFs half way in a DerbyDB from the prev Oozie release and the test runs them to completion with the new Oozie.

        Show
        tucu00 Alejandro Abdelnur added a comment - Good catch Virag, my bad here. To avoid this in the future, we should create an integration test that has a few WFs half way in a DerbyDB from the prev Oozie release and the test runs them to completion with the new Oozie.
        Hide
        virag Virag Kothari added a comment -

        Added back the inner classes from 3.2. Did some initial testing on suspend, resume, kill on 3.2 jobs after upgrade to 3.3, works fine.
        Patch on Rb: https://reviews.apache.org/r/8573

        Show
        virag Virag Kothari added a comment - Added back the inner classes from 3.2. Did some initial testing on suspend, resume, kill on 3.2 jobs after upgrade to 3.3, works fine. Patch on Rb: https://reviews.apache.org/r/8573

          People

          • Assignee:
            virag Virag Kothari
            Reporter:
            virag Virag Kothari
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development