Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Activity

      Hide
      Matt Foley added a comment -

      The underlying bug that caused the failures is fixed in HDFS-1855. This patch for HDFS-1852 subtask HDFS-1856 provides these improvements:

      • adds timeouts to all "wait for condition" loops
      • adjusts config parameters to speed up test execution a lot
      • improves log messages, clarifies code, and fixes obsolete and incorrect comments
      Show
      Matt Foley added a comment - The underlying bug that caused the failures is fixed in HDFS-1855 . This patch for HDFS-1852 subtask HDFS-1856 provides these improvements: adds timeouts to all "wait for condition" loops adjusts config parameters to speed up test execution a lot improves log messages, clarifies code, and fixes obsolete and incorrect comments
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12476971/TestDatanodeBlockScanner_tmouts_v2.patch
      against trunk revision 1095789.

      +1 @author. The patch does not contain any @author tags.

      +1 tests included. The patch appears to include 3 new or modified tests.

      +1 javadoc. The javadoc tool did not generate any warning messages.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      -1 core tests. The patch failed these core unit tests:
      org.apache.hadoop.hdfs.TestFileAppend4
      org.apache.hadoop.hdfs.TestFileConcurrentReader
      org.apache.hadoop.hdfs.TestLargeBlock
      org.apache.hadoop.hdfs.TestWriteConfigurationToDFS

      +1 contrib tests. The patch passed contrib unit tests.

      +1 system test framework. The patch passed system test framework compile.

      Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/395//testReport/
      Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/395//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
      Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/395//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12476971/TestDatanodeBlockScanner_tmouts_v2.patch against trunk revision 1095789. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.TestFileAppend4 org.apache.hadoop.hdfs.TestFileConcurrentReader org.apache.hadoop.hdfs.TestLargeBlock org.apache.hadoop.hdfs.TestWriteConfigurationToDFS +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/395//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/395//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/395//console This message is automatically generated.
      Hide
      Eli Collins added a comment -

      Updated patch that applies atop HDFS-1562.

      Show
      Eli Collins added a comment - Updated patch that applies atop HDFS-1562 .
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12477062/TestDatanodeBlockScanner_tmouts_v3.patch
      against trunk revision 1095830.

      +1 @author. The patch does not contain any @author tags.

      +1 tests included. The patch appears to include 3 new or modified tests.

      -1 patch. The patch command could not apply the patch.

      Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/405//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12477062/TestDatanodeBlockScanner_tmouts_v3.patch against trunk revision 1095830. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/405//console This message is automatically generated.
      Hide
      Matt Foley added a comment -

      Thanks Eli. We'll re-run Hudson after HDFS-1562 is committed.

      Show
      Matt Foley added a comment - Thanks Eli. We'll re-run Hudson after HDFS-1562 is committed.
      Hide
      Eli Collins added a comment -

      Attaching the patch again to trigger Hudson.

      Show
      Eli Collins added a comment - Attaching the patch again to trigger Hudson.
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12477129/TestDatanodeBlockScanner_tmouts_v4.patch
      against trunk revision 1095963.

      +1 @author. The patch does not contain any @author tags.

      +1 tests included. The patch appears to include 3 new or modified tests.

      +1 javadoc. The javadoc tool did not generate any warning messages.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      -1 core tests. The patch failed these core unit tests:
      org.apache.hadoop.hdfs.TestFileConcurrentReader

      +1 contrib tests. The patch passed contrib unit tests.

      +1 system test framework. The patch passed system test framework compile.

      Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/410//testReport/
      Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/410//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
      Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/410//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12477129/TestDatanodeBlockScanner_tmouts_v4.patch against trunk revision 1095963. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.TestFileConcurrentReader +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/410//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/410//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/410//console This message is automatically generated.
      Hide
      Eli Collins added a comment -

      +1

      The test failure is unrelated.

      Show
      Eli Collins added a comment - +1 The test failure is unrelated.
      Hide
      Eli Collins added a comment -

      I've committed this. Thanks Matt!

      Show
      Eli Collins added a comment - I've committed this. Thanks Matt!
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Hdfs-trunk-Commit #602 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/602/)
      HDFS-1856. TestDatanodeBlockScanner waits forever, errs without giving information. Contributed by Matt Foley

      Show
      Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #602 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/602/ ) HDFS-1856 . TestDatanodeBlockScanner waits forever, errs without giving information. Contributed by Matt Foley
      Hide
      Matt Foley added a comment -

      re-upload to trigger Hudson

      Show
      Matt Foley added a comment - re-upload to trigger Hudson
      Hide
      Matt Foley added a comment -

      Woops, should have refreshed my browser. Sorry for the spam.

      Show
      Matt Foley added a comment - Woops, should have refreshed my browser. Sorry for the spam.
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Hdfs-trunk #650 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/650/)

      Show
      Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #650 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/650/ )
      Hide
      Tsz Wo Nicholas Sze added a comment -

      TestDatanodeBlockScanner is failing on trunk; see build #446.

      java.util.concurrent.TimeoutException: Timed out waiting for corrupt replicas. Waiting for 1, but only found 1
      	at org.apache.hadoop.hdfs.DFSTestUtil.waitCorruptReplicas(DFSTestUtil.java:308)
      	at org.apache.hadoop.hdfs.TestDatanodeBlockScanner.blockCorruptionRecoveryPolicy(TestDatanodeBlockScanner.java:288)
      	at org.apache.hadoop.hdfs.TestDatanodeBlockScanner.testBlockCorruptionRecoveryPolicy1(TestDatanodeBlockScanner.java:236)
      
      Show
      Tsz Wo Nicholas Sze added a comment - TestDatanodeBlockScanner is failing on trunk; see build #446 . java.util.concurrent.TimeoutException: Timed out waiting for corrupt replicas. Waiting for 1, but only found 1 at org.apache.hadoop.hdfs.DFSTestUtil.waitCorruptReplicas(DFSTestUtil.java:308) at org.apache.hadoop.hdfs.TestDatanodeBlockScanner.blockCorruptionRecoveryPolicy(TestDatanodeBlockScanner.java:288) at org.apache.hadoop.hdfs.TestDatanodeBlockScanner.testBlockCorruptionRecoveryPolicy1(TestDatanodeBlockScanner.java:236)
      Hide
      Matt Foley added a comment -

      Please see HDFS-1882. This is new mis-behavior since HDFS-1052 was integrated.

      Show
      Matt Foley added a comment - Please see HDFS-1882 . This is new mis-behavior since HDFS-1052 was integrated.

        People

        • Assignee:
          Matt Foley
          Reporter:
          Matt Foley
        • Votes:
          0 Vote for this issue
          Watchers:
          0 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development