Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1856

Extract a subset of tests for smoke (DOA) validation

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Similar to that of HDFS-1199 for MapReduce.
      Adds an ability to run up to 30 minutes of the tests to 'smoke' MapReduce build i.e. find possible issues faster than the full test cycle does).

      1. MAPREDUCE-1856.patch
        4 kB
        Konstantin Boudnik
      2. MAPREDUCE-1856.patch
        4 kB
        Konstantin Boudnik
      3. MAPREDUCE-1856.patch
        4 kB
        Konstantin Boudnik
      4. MAPREDUCE-1856.patch
        4 kB
        Konstantin Boudnik
      5. MAPREDUCE-1856.patch
        4 kB
        Konstantin Boudnik
      6. MAPREDUCE-1856.patch
        4 kB
        Konstantin Boudnik

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/ )
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Konstantin Boudnik made changes -
          Resolution Fixed [ 1 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.21.0 [ 12314045 ]
          Hide
          Konstantin Boudnik added a comment -

          I have committed this.

          Show
          Konstantin Boudnik added a comment - I have committed this.
          Hide
          Jakob Homan added a comment -

          +1.

          Show
          Jakob Homan added a comment - +1.
          Hide
          Konstantin Boudnik added a comment -

          I have ran test-patch on my dev. machine

          -1 overall.
              +1 @author.  The patch does not contain any @author tags.
              -1 tests included.  The patch doesn't appear to include any new or modified tests.
                                  Please justify why no new tests are needed for this patch.
                                  Also please list what manual steps were performed to verify this patch.
              +1 javadoc.  The javadoc tool did not generate any warning messages.
              +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
              +1 findbugs.  The patch does not introduce any new Findbugs warnings.
              +1 release audit.  The applied patch does not increase the total number of release audit warnings.
          

          No tests warning is not applicable because this is the build change. It seems the patch is ready to be committed.

          Show
          Konstantin Boudnik added a comment - I have ran test-patch on my dev. machine -1 overall. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. No tests warning is not applicable because this is the build change. It seems the patch is ready to be committed.
          Hide
          Konstantin Boudnik added a comment -

          Test failures seems to be unrelated to the modification introduced by the patch: some tests have timeout'd and TestSimulatorJobTracker failing because of NPE. This is pretty hard to tell what else patches were tested at the same time for the Hudson apparently in a lot of pain for the last few days ;(

          I'm going to commit it by end of Monday unless hear otherwise.

          Show
          Konstantin Boudnik added a comment - Test failures seems to be unrelated to the modification introduced by the patch: some tests have timeout'd and TestSimulatorJobTracker failing because of NPE. This is pretty hard to tell what else patches were tested at the same time for the Hudson apparently in a lot of pain for the last few days ;( I'm going to commit it by end of Monday unless hear otherwise.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12451395/MAPREDUCE-1856.patch
          against trunk revision 984707.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/355/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/355/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/355/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/355/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12451395/MAPREDUCE-1856.patch against trunk revision 984707. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/355/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/355/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/355/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/355/console This message is automatically generated.
          Konstantin Boudnik made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Konstantin Boudnik added a comment -

          has not been picked up in 6 days. Resubmitting

          Show
          Konstantin Boudnik added a comment - has not been picked up in 6 days. Resubmitting
          Konstantin Boudnik made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Konstantin Boudnik made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Konstantin Boudnik made changes -
          Attachment MAPREDUCE-1856.patch [ 12451395 ]
          Hide
          Konstantin Boudnik added a comment -

          Updating it with more fixes as in HDFS-806

          Show
          Konstantin Boudnik added a comment - Updating it with more fixes as in HDFS-806
          Konstantin Boudnik made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Konstantin Boudnik made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Konstantin Boudnik added a comment -

          Re-submitting.

          Show
          Konstantin Boudnik added a comment - Re-submitting.
          Konstantin Boudnik made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Konstantin Boudnik added a comment -

          Looks like Hudson is stuck or something....

          Show
          Konstantin Boudnik added a comment - Looks like Hudson is stuck or something....
          Jakob Homan made changes -
          Hadoop Flags [Reviewed]
          Hide
          Jakob Homan added a comment -

          +1

          Show
          Jakob Homan added a comment - +1
          Konstantin Boudnik made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Konstantin Boudnik made changes -
          Attachment MAPREDUCE-1856.patch [ 12450030 ]
          Hide
          Konstantin Boudnik added a comment -

          Missed the test list.

          Show
          Konstantin Boudnik added a comment - Missed the test list.
          Konstantin Boudnik made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Konstantin Boudnik made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Konstantin Boudnik added a comment -

          Let's re-verify

          Show
          Konstantin Boudnik added a comment - Let's re-verify
          Konstantin Boudnik made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Konstantin Boudnik made changes -
          Attachment MAPREDUCE-1856.patch [ 12450025 ]
          Hide
          Konstantin Boudnik added a comment -

          Accounts for a corner case described in HDFS-1311

          Show
          Konstantin Boudnik added a comment - Accounts for a corner case described in HDFS-1311
          Hide
          Konstantin Boudnik added a comment -
          • Release audit warning is caused by absence of Apache license in test list file. We don't have them there as a general rule so this warning can be safely disregarded.
          • Test failures are unrelated: one is 7 days old and another is cause by incomplete clean-up or a similar issues.
          Show
          Konstantin Boudnik added a comment - Release audit warning is caused by absence of Apache license in test list file. We don't have them there as a general rule so this warning can be safely disregarded. Test failures are unrelated: one is 7 days old and another is cause by incomplete clean-up or a similar issues.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12448588/MAPREDUCE-1856.patch
          against trunk revision 959867.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 16 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings).

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/284/testReport/
          Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/284/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/284/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/284/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/284/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448588/MAPREDUCE-1856.patch against trunk revision 959867. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 16 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings). -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/284/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/284/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/284/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/284/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/284/console This message is automatically generated.
          Konstantin Boudnik made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Konstantin Boudnik made changes -
          Attachment MAPREDUCE-1856.patch [ 12448588 ]
          Hide
          Konstantin Boudnik added a comment -

          Oops

          Show
          Konstantin Boudnik added a comment - Oops
          Konstantin Boudnik made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12448470/MAPREDUCE-1856.patch
          against trunk revision 959806.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 15 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          -1 javac. The patch appears to cause tar ant target to fail.

          -1 findbugs. The patch appears to cause Findbugs to fail.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/587/testReport/
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/587/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/587/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448470/MAPREDUCE-1856.patch against trunk revision 959806. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. -1 findbugs. The patch appears to cause Findbugs to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/587/testReport/ Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/587/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/587/console This message is automatically generated.
          Konstantin Boudnik made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Konstantin Boudnik made changes -
          Attachment MAPREDUCE-1856.patch [ 12448470 ]
          Hide
          Konstantin Boudnik added a comment -

          Refitting the patch for trunk.

          Show
          Konstantin Boudnik added a comment - Refitting the patch for trunk.
          Konstantin Boudnik made changes -
          Link This issue is related to HADOOP-6810 [ HADOOP-6810 ]
          Konstantin Boudnik made changes -
          Field Original Value New Value
          Attachment MAPREDUCE-1856.patch [ 12446822 ]
          Hide
          Konstantin Boudnik added a comment -

          This patch introduces an extra target which allow to execute all tests excluding commit and smoke sets. Also about 15 minutes worth of tests is added.

          Show
          Konstantin Boudnik added a comment - This patch introduces an extra target which allow to execute all tests excluding commit and smoke sets. Also about 15 minutes worth of tests is added.
          Konstantin Boudnik created issue -

            People

            • Assignee:
              Konstantin Boudnik
              Reporter:
              Konstantin Boudnik
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development