Details

    • Type: Task Task
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 3.2.0, 3.3.0, trunk
    • Fix Version/s: 3.3.2
    • Component/s: None
    • Labels:
      None

      Description

      Would be nice to have support for Hadoop 1.1.1.

      1. OOZIE-1088-trunk.patch
        3 kB
        Harsh J
      2. OOZIE-816.patch
        3 kB
        Harsh J
      3. OOZIE-816.patch
        3 kB
        Harsh J
      4. OOZIE-816.patch
        3 kB
        Harsh J
      5. OOZIE-816.patch
        3 kB
        Grant Ingersoll
      6. OOZIE-816-3.2.0.patch
        7 kB
        zhu jin wei

        Issue Links

          Activity

          Rohini Palaniswamy made changes -
          Fix Version/s trunk [ 12323240 ]
          Robert Kanter made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Robert Kanter added a comment -

          Closing issue; Oozie 3.3.2 is released

          Show
          Robert Kanter added a comment - Closing issue; Oozie 3.3.2 is released
          Robert Kanter made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Robert Kanter added a comment -

          Resolving as "Fixed" because this already is in branch-3.3 and trunk. It won't go into branch-3.2 because we're not making any more releases from that branch.

          Show
          Robert Kanter added a comment - Resolving as "Fixed" because this already is in branch-3.3 and trunk. It won't go into branch-3.2 because we're not making any more releases from that branch.
          Robert Kanter made changes -
          Fix Version/s 3.3.2 [ 12323993 ]
          Hide
          Harsh J added a comment -

          Hi Zhu,

          Haven't heard back on your 3.3 backport request (above comment)?

          Show
          Harsh J added a comment - Hi Zhu, Haven't heard back on your 3.3 backport request (above comment)?
          Hide
          Harsh J added a comment -

          Hi,

          AFAIK, we are no longer keen on maintaining the 3.2-branch now that 3.3 is out (there have been no commits on the 3.2 branch in over 3 months). Would targeting a future 3.3.x release be OK instead?

          Show
          Harsh J added a comment - Hi, AFAIK, we are no longer keen on maintaining the 3.2-branch now that 3.3 is out (there have been no commits on the 3.2 branch in over 3 months). Would targeting a future 3.3.x release be OK instead?
          Hide
          Hadoop QA added a comment -

          Testing JIRA OOZIE-816

          Cleaning local svn workspace

          ----------------------------

          -1 Patch failed to apply to head of branch

          ----------------------------

          Show
          Hadoop QA added a comment - Testing JIRA OOZIE-816 Cleaning local svn workspace ---------------------------- -1 Patch failed to apply to head of branch ----------------------------
          zhu jin wei made changes -
          Status Reopened [ 4 ] Patch Available [ 10002 ]
          Hide
          zhu jin wei added a comment -

          Hi,Harsh,I reopen this jira, because I want to upgrade branch oozie3.2.0's hadoop version into 1.1.1.

          Show
          zhu jin wei added a comment - Hi,Harsh,I reopen this jira, because I want to upgrade branch oozie3.2.0's hadoop version into 1.1.1.
          zhu jin wei made changes -
          Assignee zhu jin wei [ zhujinwei ] Harsh J [ qwertymaniac ]
          zhu jin wei made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          zhu jin wei made changes -
          Attachment OOZIE-816-3.2.0.patch [ 12559810 ]
          zhu jin wei made changes -
          Affects Version/s 3.2.0 [ 12320149 ]
          Affects Version/s trunk [ 12323240 ]
          Hide
          zhu jin wei added a comment -

          Hi, Harsh, I have upgraded branch oozie3.2.0's hadoop version into 1.1.1, I want to upload my patch this jira too.

          And I have run "mvn test", all the UT passed. I also want to run "testpatch" , but in oozie3.2.0 source code , there is no testpatch script in "bin" directory. Do you know if I can use trunk's testpatch script to test my patch?

          Another thing, do you know how to format patch or do you have fromat template ?

          Show
          zhu jin wei added a comment - Hi, Harsh, I have upgraded branch oozie3.2.0's hadoop version into 1.1.1, I want to upload my patch this jira too. And I have run "mvn test", all the UT passed. I also want to run "testpatch" , but in oozie3.2.0 source code , there is no testpatch script in "bin" directory. Do you know if I can use trunk's testpatch script to test my patch? Another thing, do you know how to format patch or do you have fromat template ?
          Hide
          zhu jin wei added a comment -

          Thanks, Harsh. I think the fail tests may be flaky,too.

          Show
          zhu jin wei added a comment - Thanks, Harsh. I think the fail tests may be flaky,too.
          Harsh J made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s trunk [ 12323240 ]
          Resolution Fixed [ 1 ]
          Hide
          Harsh J added a comment -

          Thanks zhu jin wei, I've committed this to trunk.

          Show
          Harsh J added a comment - Thanks zhu jin wei , I've committed this to trunk.
          Hide
          Harsh J added a comment -

          Unclear what caused these two failures (log does not seem to indicate an issue clearly).

          Running them locally does pass, so tests may be flaky:

          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Running org.apache.oozie.service.TestPauseTransitService
          Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.473 sec
          Running org.apache.oozie.service.TestStatusTransitService
          Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 143.223 sec
          
          Results :
          
          Tests run: 37, Failures: 0, Errors: 0, Skipped: 0
          

          Will go ahead and commit and we can investigate these failures if they appear over all builds again.

          Show
          Harsh J added a comment - Unclear what caused these two failures (log does not seem to indicate an issue clearly). Running them locally does pass, so tests may be flaky: ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.oozie.service.TestPauseTransitService Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.473 sec Running org.apache.oozie.service.TestStatusTransitService Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 143.223 sec Results : Tests run: 37, Failures: 0, Errors: 0, Skipped: 0 Will go ahead and commit and we can investigate these failures if they appear over all builds again.
          Hide
          Hadoop QA added a comment -

          Testing JIRA OOZIE-816

          Cleaning local svn workspace

          ----------------------------

          +1 PATCH_APPLIES
          +1 CLEAN
          -1 RAW_PATCH_ANALYSIS
          . +1 the patch does not introduce any @author tags
          . +1 the patch does not introduce any tabs
          . +1 the patch does not introduce any trailing spaces
          . +1 the patch does not introduce any line longer than 132
          . -1 the patch does not add/modify any testcase
          +1 RAT
          . +1 the patch does not seem to introduce new RAT warnings
          +1 JAVADOC
          . +1 the patch does not seem to introduce new Javadoc warnings
          +1 COMPILE
          . +1 HEAD compiles
          . +1 patch compiles
          . +1 the patch does not seem to introduce new javac warnings
          +1 BACKWARDS_COMPATIBILITY
          . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
          . +1 the patch does not modify JPA files
          -1 TESTS
          . Tests run: 925
          . Tests failed: 2
          . Tests errors: 0

          . The patch failed the following testcases:

          . testBundleStatusTransitServiceKilled1(org.apache.oozie.service.TestStatusTransitService)
          . testUnpauseBundleAndCoordinator(org.apache.oozie.service.TestPauseTransitService)

          +1 DISTRO
          . +1 distro tarball builds with the patch

          ----------------------------
          -1 Overall result, please check the reported -1(s)

          The full output of the test-patch run is available at

          . https://builds.apache.org/job/oozie-trunk-precommit-build/229/

          Show
          Hadoop QA added a comment - Testing JIRA OOZIE-816 Cleaning local svn workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . -1 the patch does not add/modify any testcase +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files -1 TESTS . Tests run: 925 . Tests failed: 2 . Tests errors: 0 . The patch failed the following testcases: . testBundleStatusTransitServiceKilled1(org.apache.oozie.service.TestStatusTransitService) . testUnpauseBundleAndCoordinator(org.apache.oozie.service.TestPauseTransitService) +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/229/
          Harsh J made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Harsh J made changes -
          Attachment OOZIE-816.patch [ 12555740 ]
          Hide
          Harsh J added a comment -

          New, proper SVN-applicable patch.

          Show
          Harsh J added a comment - New, proper SVN-applicable patch.
          Harsh J made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Harsh J added a comment -

          Patch didn't have --no-prefix. Re-upping.

          Show
          Harsh J added a comment - Patch didn't have --no-prefix. Re-upping.
          Hide
          Hadoop QA added a comment -

          Testing JIRA OOZIE-816

          Cleaning local svn workspace

          ----------------------------

          -1 Patch failed to apply to head of branch

          ----------------------------

          Show
          Hadoop QA added a comment - Testing JIRA OOZIE-816 Cleaning local svn workspace ---------------------------- -1 Patch failed to apply to head of branch ----------------------------
          Harsh J made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Harsh J made changes -
          Attachment OOZIE-816.patch [ 12555737 ]
          Hide
          Harsh J added a comment -

          Here's a new patch that changes Zhu's bump to 1.0.4 to 1.1.1 which Apache Hadoop's just announced as a stable release upstream (since we're upgrading trunk, we could as well go to 1.1.1).

          Also re-triggering tests.

          Show
          Harsh J added a comment - Here's a new patch that changes Zhu's bump to 1.0.4 to 1.1.1 which Apache Hadoop's just announced as a stable release upstream (since we're upgrading trunk, we could as well go to 1.1.1). Also re-triggering tests.
          Harsh J made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Harsh J made changes -
          Description Would be nice to have support for Hadoop 1.0.3. Would be nice to have support for Hadoop 1.1.1.
          Harsh J made changes -
          Summary Add Support for Hadoop 1.0.3 Add Support for Hadoop 1.1.1
          Hide
          zhu jin wei added a comment -

          Hi,Harsh, could you help to re-run the test-patch ?

          Show
          zhu jin wei added a comment - Hi,Harsh, could you help to re-run the test-patch ?
          zhu jin wei made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          zhu jin wei made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          zhu jin wei added a comment -

          I have only run TestJavaActionExecutor, and it passed. I think we need to run it again with OOZIE-1088-trunk.patch

          Show
          zhu jin wei added a comment - I have only run TestJavaActionExecutor, and it passed. I think we need to run it again with OOZIE-1088 -trunk.patch
          Hide
          Hadoop QA added a comment -

          Testing JIRA OOZIE-816

          Cleaning local svn workspace

          ----------------------------

          +1 PATCH_APPLIES
          +1 CLEAN
          -1 RAW_PATCH_ANALYSIS
          . +1 the patch does not introduce any @author tags
          . +1 the patch does not introduce any tabs
          . +1 the patch does not introduce any trailing spaces
          . +1 the patch does not introduce any line longer than 132
          . -1 the patch does not add/modify any testcase
          +1 RAT
          . +1 the patch does not seem to introduce new RAT warnings
          +1 JAVADOC
          . +1 the patch does not seem to introduce new Javadoc warnings
          +1 COMPILE
          . +1 HEAD compiles
          . +1 patch compiles
          . +1 the patch does not seem to introduce new javac warnings
          +1 BACKWARDS_COMPATIBILITY
          . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
          . +1 the patch does not modify JPA files
          -1 TESTS
          . Tests run: 922
          . Tests failed: 1
          . Tests errors: 0

          . The patch failed the following testcases:

          . testRecovery(org.apache.oozie.action.hadoop.TestJavaActionExecutor)

          +1 DISTRO
          . +1 distro tarball builds with the patch

          ----------------------------
          -1 Overall result, please check the reported -1(s)

          The full output of the test-patch run is available at

          . https://builds.apache.org/job/oozie-trunk-precommit-build/215/

          Show
          Hadoop QA added a comment - Testing JIRA OOZIE-816 Cleaning local svn workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . -1 the patch does not add/modify any testcase +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files -1 TESTS . Tests run: 922 . Tests failed: 1 . Tests errors: 0 . The patch failed the following testcases: . testRecovery(org.apache.oozie.action.hadoop.TestJavaActionExecutor) +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/215/
          Harsh J made changes -
          Assignee zhu jin wei [ zhujinwei ]
          Hide
          Harsh J added a comment -

          Assigning to Zhu Jin Wei.

          Show
          Harsh J added a comment - Assigning to Zhu Jin Wei.
          Harsh J made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 3.3.0 [ 12320755 ]
          Harsh J made changes -
          Attachment OOZIE-1088-trunk.patch [ 12555332 ]
          Hide
          Harsh J added a comment -

          Attaching Zhu's patch from dupe OOZIE-1088 issue here.

          Show
          Harsh J added a comment - Attaching Zhu's patch from dupe OOZIE-1088 issue here.
          Harsh J made changes -
          Assignee Harsh J [ qwertymaniac ]
          Hide
          Harsh J added a comment -

          Hi,

          Sorry about the assigned part, I unassigned myself now. Please do go forward with your patch which you have already tested

          If 1.0.4 is more preferable, then that is fine with me as well. I am not quite sure why 1.1.1 is marked unstable, cause its a bugfix revision over 1.1.0 anyway.

          Show
          Harsh J added a comment - Hi, Sorry about the assigned part, I unassigned myself now. Please do go forward with your patch which you have already tested If 1.0.4 is more preferable, then that is fine with me as well. I am not quite sure why 1.1.1 is marked unstable, cause its a bugfix revision over 1.1.0 anyway.
          Hide
          zhu jin wei added a comment -

          Hi,Harsh,Since you are the assignee for this jira, could you update your patch according to Alejandro's method?

          Show
          zhu jin wei added a comment - Hi,Harsh,Since you are the assignee for this jira, could you update your patch according to Alejandro's method?
          Hide
          zhu jin wei added a comment -

          Hi,Harsh, as far as know, 1.1.1 is a beta release , and comparing with 1.1.1, 1.0.4 is a more stable version.
          And, I have also tested 1.0.4, all oozie UI passed. I have upload my patch in jira 1088.
          So,I suggest we can upgrade to 1.0.4 in main pom.xml
          MeanWhile, we can continue to support 1.0.1 ,1.0.3 adn 1.1.1 by using Alejandro's method.

          Show
          zhu jin wei added a comment - Hi,Harsh, as far as know, 1.1.1 is a beta release , and comparing with 1.1.1, 1.0.4 is a more stable version. And, I have also tested 1.0.4, all oozie UI passed. I have upload my patch in jira 1088. So,I suggest we can upgrade to 1.0.4 in main pom.xml MeanWhile, we can continue to support 1.0.1 ,1.0.3 adn 1.1.1 by using Alejandro's method.
          Hide
          Harsh J added a comment -

          Apache Hadoop got a green for the 1.1.1 release yesterday, so we can wait and bump up the trunk to that.

          Of note is that HBase's next version (0.96, no ETA yet though) would mark a strong move to the 2.x line of releases and begin to discontinue proper 1.x support.

          Show
          Harsh J added a comment - Apache Hadoop got a green for the 1.1.1 release yesterday, so we can wait and bump up the trunk to that. Of note is that HBase's next version (0.96, no ETA yet though) would mark a strong move to the 2.x line of releases and begin to discontinue proper 1.x support.
          Harsh J made changes -
          Link This issue is duplicated by OOZIE-1088 [ OOZIE-1088 ]
          Harsh J made changes -
          Attachment OOZIE-816.patch [ 12536439 ]
          Hide
          Harsh J added a comment -

          Patch for initial review. Switches all targets up to 1.0.3.

          Still running tests, so not marking as available.

          Show
          Harsh J added a comment - Patch for initial review. Switches all targets up to 1.0.3. Still running tests, so not marking as available.
          Harsh J made changes -
          Summary Add Support for Hadoop 1.0.2 Add Support for Hadoop 1.0.3
          Assignee Harsh J [ qwertymaniac ]
          Description Would be nice to have support for Hadoop 1.0.2. Would be nice to have support for Hadoop 1.0.3.
          Hide
          Harsh J added a comment -

          Since 1.0.3 is out, lets target that.

          I suspect a 1.0.4 may arise due to a critical HDFS issue, but I do not know any formal announcements for it yet.

          Sending patch after testing shortly.

          Show
          Harsh J added a comment - Since 1.0.3 is out, lets target that. I suspect a 1.0.4 may arise due to a critical HDFS issue, but I do not know any formal announcements for it yet. Sending patch after testing shortly.
          Alejandro Abdelnur made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Alejandro Abdelnur added a comment -

          canceling patch as it is not correct.

          Grant, please let us know if you'll be taking care of a new patch. The new patch should just rev up the hadoop version in the hadoop-1 and hadoop-test-1 POMs and set the new default in the main POM. thx

          Show
          Alejandro Abdelnur added a comment - canceling patch as it is not correct. Grant, please let us know if you'll be taking care of a new patch. The new patch should just rev up the hadoop version in the hadoop-1 and hadoop-test-1 POMs and set the new default in the main POM. thx
          Hide
          Alejandro Abdelnur added a comment -

          @Grant, (sorry for the delay on this one), the only thing we should do to support Hadoop 1.0.2 should be creating under hadooplibs the corresponding hadoop-1_0_1 and hadoop-test-1_0_2 directories. The easiest way it to copy the POMs from 1.0.1 and refer to the new version. And of course wire the new modules to the hadooplibs POM.

          Please let us know if you'll take care of it. thx

          Show
          Alejandro Abdelnur added a comment - @Grant, (sorry for the delay on this one), the only thing we should do to support Hadoop 1.0.2 should be creating under hadooplibs the corresponding hadoop-1_0_1 and hadoop-test-1_0_2 directories. The easiest way it to copy the POMs from 1.0.1 and refer to the new version. And of course wire the new modules to the hadooplibs POM. Please let us know if you'll take care of it. thx
          Hide
          Mohammad Kamrul Islam added a comment -

          Hi Grant,
          Welcome back.
          would you please create review request using reviews.apache.org. We mostly use that for patch review.

          Regards,
          Mohammad

          Show
          Mohammad Kamrul Islam added a comment - Hi Grant, Welcome back. would you please create review request using reviews.apache.org. We mostly use that for patch review. Regards, Mohammad
          Grant Ingersoll made changes -
          Attachment OOZIE-816.patch [ 12523282 ]
          Hide
          Grant Ingersoll added a comment -

          Totally new to Oozie code (have used it before though), so not sure if this is right, but mimicked what was already in place. Probably needs more work.

          Show
          Grant Ingersoll added a comment - Totally new to Oozie code (have used it before though), so not sure if this is right, but mimicked what was already in place. Probably needs more work.
          Grant Ingersoll made changes -
          Field Original Value New Value
          Status Open [ 1 ] Patch Available [ 10002 ]
          Grant Ingersoll created issue -

            People

            • Assignee:
              Harsh J
              Reporter:
              Grant Ingersoll
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development