Hadoop Common
  1. Hadoop Common
  2. HADOOP-6786

test-patch needs to verify Herriot integrity

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.1
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Whenever a new patch is submitted for verification test-patch process has to make sure that none of Herriot bindings were broken.

      1. HADOOP-6786.patch
        2 kB
        Konstantin Boudnik
      2. HADOOP-6786.patch
        2 kB
        Konstantin Boudnik
      3. HADOOP-6786.patch
        2 kB
        Konstantin Boudnik

        Issue Links

          Activity

          Hide
          Konstantin Boudnik added a comment -

          To achieve this test-patch needs to execute one more ant target inject-system-faults which will perform aspects weaving and will guarantee that changes in the source code doesn't brake the integrity of the test code.

          Show
          Konstantin Boudnik added a comment - To achieve this test-patch needs to execute one more ant target inject-system-faults which will perform aspects weaving and will guarantee that changes in the source code doesn't brake the integrity of the test code.
          Hide
          Konstantin Boudnik added a comment -

          The patch introduces new check which makes sure that system test framework bindings aren't broken by running inject-system-faults target. This is a simple compilation + aspect weaving and doesn't execute any additional tests.

          Show
          Konstantin Boudnik added a comment - The patch introduces new check which makes sure that system test framework bindings aren't broken by running inject-system-faults target. This is a simple compilation + aspect weaving and doesn't execute any additional tests.
          Hide
          Konstantin Boudnik added a comment -

          This shouldn't be committed before these JIRAs are integrated because they introduce inject-system-faults target.

          Show
          Konstantin Boudnik added a comment - This shouldn't be committed before these JIRAs are integrated because they introduce inject-system-faults target.
          Hide
          Konstantin Boudnik added a comment -

          The patch can be verified in Common because Herriot framework is already integrated here.
          Also, I have ran the new version of test-patch.sh locally and I can see that the required target is being invoked and new check's result is printed at the end of the process:

              +1 system tests framework.  The patch passed system tests framework compile.
          
          Show
          Konstantin Boudnik added a comment - The patch can be verified in Common because Herriot framework is already integrated here. Also, I have ran the new version of test-patch.sh locally and I can see that the required target is being invoked and new check's result is printed at the end of the process: +1 system tests framework. The patch passed system tests framework compile.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12445609/HADOOP-6786.patch
          against trunk revision 948573.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/549/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/549/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/549/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/549/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12445609/HADOOP-6786.patch against trunk revision 948573. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/549/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/549/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/549/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/549/console This message is automatically generated.
          Hide
          Konstantin Boudnik added a comment -

          A review, please? All blockers were resolved.

          Show
          Konstantin Boudnik added a comment - A review, please? All blockers were resolved.
          Hide
          Giridharan Kesavan added a comment -

          + $PS auxwww | $GREP HadoopPatchProcess | /usr/bin/nawk '

          Unknown macro: {print $2}

          ' | /usr/bin/xargs -t -I {} /usr/bin/kill -9 {} > /dev/null

          all the patch test builds run on ubuntu machines and kill cmd seem to reside in /bin/kill

          Show
          Giridharan Kesavan added a comment - + $PS auxwww | $GREP HadoopPatchProcess | /usr/bin/nawk ' Unknown macro: {print $2} ' | /usr/bin/xargs -t -I {} /usr/bin/kill -9 {} > /dev/null all the patch test builds run on ubuntu machines and kill cmd seem to reside in /bin/kill
          Hide
          Konstantin Boudnik added a comment -

          all the patch test builds run on ubuntu machines and kill cmd seem to reside in /bin/kill

          I wonder how it works in two other places (for core and contrib tests) then?

          Show
          Konstantin Boudnik added a comment - all the patch test builds run on ubuntu machines and kill cmd seem to reside in /bin/kill I wonder how it works in two other places (for core and contrib tests) then?
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I may have missed something:

          • In the code below, why setting PreTestTarget to a empty string?
            +  PreTestTarget=""
            +
            +  echo "$ANT_HOME/bin/ant -Dversion="${VERSION}" -DHadoopPatchProcess= -Dtest.junit.output.format=xml -Dtest.output=yes -Dcompile.c++=yes -Dforrest.home=$FORREST_HOME -Djava5.home=$JAVA5_HOME $PreTestTarget inject-system-faults"
            +  $ANT_HOME/bin/ant -Dversion="${VERSION}" -DHadoopPatchProcess= -Dtest.junit.output.format=xml -Dtest.output=yes -Dcompile.c++=yes -Dforrest.home=$FORREST_HOME -Djava5.home=$JAVA5_HOME $PreTestTarget inject-system-faults
            
          • The inject-system-faults target is not found in build.xml. Where is it defined?
          Show
          Tsz Wo Nicholas Sze added a comment - I may have missed something: In the code below, why setting PreTestTarget to a empty string? + PreTestTarget="" + + echo "$ANT_HOME/bin/ant -Dversion=" ${VERSION} " -DHadoopPatchProcess= -Dtest.junit.output.format=xml -Dtest.output=yes -Dcompile.c++=yes -Dforrest.home=$FORREST_HOME -Djava5.home=$JAVA5_HOME $PreTestTarget inject-system-faults" + $ANT_HOME/bin/ant -Dversion= "${VERSION}" -DHadoopPatchProcess= -Dtest.junit.output.format=xml -Dtest.output=yes -Dcompile.c++=yes -Dforrest.home=$FORREST_HOME -Djava5.home=$JAVA5_HOME $PreTestTarget inject-system-faults The inject-system-faults target is not found in build.xml. Where is it defined?
          Hide
          Konstantin Boudnik added a comment -

          PreTestTarget is a garbage I have missed out. Removed.
          The target inject-system-faults is defined in src/test/aop/build/aop.xml

          Show
          Konstantin Boudnik added a comment - PreTestTarget is a garbage I have missed out. Removed. The target inject-system-faults is defined in src/test/aop/build/aop.xml
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I think we need

            (( RESULT = RESULT + $? ))
          

          after calling checkInjectSystemFaults.

          Show
          Tsz Wo Nicholas Sze added a comment - I think we need (( RESULT = RESULT + $? )) after calling checkInjectSystemFaults.
          Hide
          Konstantin Boudnik added a comment -

          Thanks for catching it up, Nicholas. (note to self: there's no 'simple' patches, damn it). Attaching new version and re-submitting the patch.

          Show
          Konstantin Boudnik added a comment - Thanks for catching it up, Nicholas. (note to self: there's no 'simple' patches, damn it). Attaching new version and re-submitting the patch.
          Hide
          Konstantin Boudnik added a comment -

          Seems to be stuck. resubmitting

          Show
          Konstantin Boudnik added a comment - Seems to be stuck. resubmitting
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 the new patch looks good.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 the new patch looks good.
          Hide
          Konstantin Boudnik added a comment -

          Hudson doesn't pick it up even after the queue has been cleaned... One more

          Show
          Konstantin Boudnik added a comment - Hudson doesn't pick it up even after the queue has been cleaned... One more
          Hide
          Konstantin Boudnik added a comment -

          No luck with Hudson.

          Show
          Konstantin Boudnik added a comment - No luck with Hudson.
          Hide
          Konstantin Boudnik added a comment -

          Hudson seems to be not at his best for the last a couple of weeks. I have ran test-patch.sh on my dev. machine.

          -1 overall.
          
              +1 @author.  The patch does not contain any @author tags.
          
              +1 tests included.  The patch appears to include 4 new or modified tests.
          
              -1 javadoc.  The javadoc tool appears to have generated 1 warning messages.
          
              +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
          
              +1 findbugs.  The patch does not introduce any new Findbugs warnings.
          
              +1 release audit.  The applied patch does not increase the total number of release audit warnings.
          

          The javadoc warning is apparently not connected to the patch itself. The only warnings I can see are in security related classes.

          Show
          Konstantin Boudnik added a comment - Hudson seems to be not at his best for the last a couple of weeks. I have ran test-patch.sh on my dev. machine. -1 overall. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. The javadoc warning is apparently not connected to the patch itself. The only warnings I can see are in security related classes.
          Hide
          Konstantin Boudnik added a comment -

          I'll be committing this by end of today or Monday if there's no objections.

          Show
          Konstantin Boudnik added a comment - I'll be committing this by end of today or Monday if there's no objections.
          Hide
          Konstantin Boudnik added a comment -

          I have committed this to 0.21 and trunk.

          Show
          Konstantin Boudnik added a comment - I have committed this to 0.21 and trunk.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #368 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/368/)
          HADOOP-6786. test-patch needs to verify Herriot integrity. Contributed by Konstantin Boudnik

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #368 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/368/ ) HADOOP-6786 . test-patch needs to verify Herriot integrity. Contributed by Konstantin Boudnik
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #377 (See https://hudson.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/377/)
          HADOOP-6786. test-patch needs to verify Herriot integrity. Contributed by Konstantin Boudnik

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #377 (See https://hudson.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/377/ ) HADOOP-6786 . test-patch needs to verify Herriot integrity. Contributed by Konstantin Boudnik
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #438 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/438/)
          HADOOP-6786. test-patch needs to verify Herriot integrity. Contributed by Konstantin Boudnik

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #438 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/438/ ) HADOOP-6786 . test-patch needs to verify Herriot integrity. Contributed by Konstantin Boudnik
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/ )

            People

            • Assignee:
              Konstantin Boudnik
              Reporter:
              Konstantin Boudnik
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development