HBase
  1. HBase
  2. HBASE-5872

Improve hadoopqa script to include checks for hadoop 0.23 build

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.95.2
    • Fix Version/s: 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      There have been a few patches that have made it into hbase trunk that have broken the compile of hbase against hadoop 0.23.x, without being known for a few days.

      We could have the bot do a few things:
      1) verify that patch compiles against hadoop 23
      2) verify that unit tests pass against hadoop 23

      1. hbase-5872-part2.patch
        0.7 kB
        Jonathan Hsieh
      2. hbase-5872.patch
        2 kB
        Jonathan Hsieh

        Activity

        Hide
        Jonathan Hsieh added a comment -

        Since #2 can be time consuming, I'm thinking #1 may be sufficient.

        Show
        Jonathan Hsieh added a comment - Since #2 can be time consuming, I'm thinking #1 may be sufficient.
        Hide
        Jonathan Hsieh added a comment -

        Also wondering if we should include the security profile for runs to verify patches don't break compiling them and that they execute the security related unit tests.

        Show
        Jonathan Hsieh added a comment - Also wondering if we should include the security profile for runs to verify patches don't break compiling them and that they execute the security related unit tests.
        Hide
        Ted Yu added a comment -
        +  echo "$MVN clean test -DskipTests -D${PROJECT_NAME}PatchProcess > $PATCH_DIR/trunkJavacWarnings.txt 2>&1"
        

        I am confused by the above: if tests are skipped, why is test target specified ?

        Show
        Ted Yu added a comment - + echo "$MVN clean test -DskipTests -D${PROJECT_NAME}PatchProcess > $PATCH_DIR/trunkJavacWarnings.txt 2>&1" I am confused by the above: if tests are skipped, why is test target specified ?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12524093/hbase-5872.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.TestRegionRebalancing
        org.apache.hadoop.hbase.replication.TestReplication

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1636//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1636//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1636//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12524093/hbase-5872.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.TestRegionRebalancing org.apache.hadoop.hbase.replication.TestReplication Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1636//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1636//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1636//console This message is automatically generated.
        Hide
        Ted Yu added a comment -

        Actually https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-Hadoop-23 is built daily.
        We just need to observe the compilation error there.

        Show
        Ted Yu added a comment - Actually https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-Hadoop-23 is built daily. We just need to observe the compilation error there.
        Hide
        Jonathan Hsieh added a comment -

        @Zhihong When I tested the script I notice that the unit tests (where there can be compilation errors) are not compiled unless the test phase is included. So, 'mvn test -DskipTests' compiles the tests but does not run them which is what I wanted. (I had added a comment about core and tests compilation).

        Show
        Jonathan Hsieh added a comment - @Zhihong When I tested the script I notice that the unit tests (where there can be compilation errors) are not compiled unless the test phase is included. So, 'mvn test -DskipTests' compiles the tests but does not run them which is what I wanted. (I had added a comment about core and tests compilation).
        Hide
        Jonathan Hsieh added a comment -

        The goal I had in mind for this patch is to get compilation problems caught before commit. Waiting for the daily build assumes that it is committed already.

        Show
        Jonathan Hsieh added a comment - The goal I had in mind for this patch is to get compilation problems caught before commit. Waiting for the daily build assumes that it is committed already.
        Hide
        Ted Yu added a comment -

        Thanks for the explanation, Jon.

        I think the patch cannot be tested by Hadoop QA. Otherwise the QA report should include have contained compilation error instead of failed tests.
        I am fine with checking in the patch after HBASE-5870 is fixed - otherwise all Hadoop QA reports would only contain compilation error

        Show
        Ted Yu added a comment - Thanks for the explanation, Jon. I think the patch cannot be tested by Hadoop QA. Otherwise the QA report should include have contained compilation error instead of failed tests. I am fine with checking in the patch after HBASE-5870 is fixed - otherwise all Hadoop QA reports would only contain compilation error
        Hide
        stack added a comment -

        +1 on option #1. +1 on the patch. +1 on what Ted says above that it'd be best if compile against 0.23 succeeded before committing this.

        Show
        stack added a comment - +1 on option #1. +1 on the patch. +1 on what Ted says above that it'd be best if compile against 0.23 succeeded before committing this.
        Hide
        Jonathan Hsieh added a comment -

        Thanks for reviews and Ted and Stack.

        Committed to trunk/0.96.

        Show
        Jonathan Hsieh added a comment - Thanks for reviews and Ted and Stack. Committed to trunk/0.96.
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2820 (See https://builds.apache.org/job/HBase-TRUNK/2820/)
        HBASE-5872 Improve hadoopqa script to include checks for hadoop 0.23 build (Revision 1331143)

        Result = SUCCESS
        jmhsieh :
        Files :

        • /hbase/trunk/dev-support/test-patch.sh
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2820 (See https://builds.apache.org/job/HBase-TRUNK/2820/ ) HBASE-5872 Improve hadoopqa script to include checks for hadoop 0.23 build (Revision 1331143) Result = SUCCESS jmhsieh : Files : /hbase/trunk/dev-support/test-patch.sh
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-security #186 (See https://builds.apache.org/job/HBase-TRUNK-security/186/)
        HBASE-5872 Improve hadoopqa script to include checks for hadoop 0.23 build (Revision 1331143)

        Result = SUCCESS
        jmhsieh :
        Files :

        • /hbase/trunk/dev-support/test-patch.sh
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-security #186 (See https://builds.apache.org/job/HBase-TRUNK-security/186/ ) HBASE-5872 Improve hadoopqa script to include checks for hadoop 0.23 build (Revision 1331143) Result = SUCCESS jmhsieh : Files : /hbase/trunk/dev-support/test-patch.sh
        Hide
        Jonathan Hsieh added a comment -

        An addendum that makes the success/failure message explicit and more informative.

        Show
        Jonathan Hsieh added a comment - An addendum that makes the success/failure message explicit and more informative.
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2821 (See https://builds.apache.org/job/HBase-TRUNK/2821/)
        HBASE-5872 ADDENDUM - improve hadoop 23 compile sucess/fail message (Revision 1331248)

        Result = FAILURE
        jmhsieh :
        Files :

        • /hbase/trunk/dev-support/test-patch.sh
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2821 (See https://builds.apache.org/job/HBase-TRUNK/2821/ ) HBASE-5872 ADDENDUM - improve hadoop 23 compile sucess/fail message (Revision 1331248) Result = FAILURE jmhsieh : Files : /hbase/trunk/dev-support/test-patch.sh
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-security #187 (See https://builds.apache.org/job/HBase-TRUNK-security/187/)
        HBASE-5872 ADDENDUM - improve hadoop 23 compile sucess/fail message (Revision 1331248)

        Result = SUCCESS
        jmhsieh :
        Files :

        • /hbase/trunk/dev-support/test-patch.sh
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-security #187 (See https://builds.apache.org/job/HBase-TRUNK-security/187/ ) HBASE-5872 ADDENDUM - improve hadoop 23 compile sucess/fail message (Revision 1331248) Result = SUCCESS jmhsieh : Files : /hbase/trunk/dev-support/test-patch.sh
        Hide
        stack added a comment -

        Marking closed.

        Show
        stack added a comment - Marking closed.

          People

          • Assignee:
            Jonathan Hsieh
            Reporter:
            Jonathan Hsieh
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development