Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8772

Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.3, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None

      Description

      https://builds.apache.org/job/PreCommit-HDFS-Build/11596/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11598/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11599/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11600/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11606/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11608/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11612/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11618/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11650/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11655/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11659/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11663/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11664/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11667/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11669/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11676/testReport/
      https://builds.apache.org/job/PreCommit-HDFS-Build/11677/testReport/

      java.lang.AssertionError: expected:<0> but was:<4>
      	at org.junit.Assert.fail(Assert.java:88)
      	at org.junit.Assert.failNotEquals(Assert.java:743)
      	at org.junit.Assert.assertEquals(Assert.java:118)
      	at org.junit.Assert.assertEquals(Assert.java:555)
      	at org.junit.Assert.assertEquals(Assert.java:542)
      	at org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot.testDatanodeRestarts(TestStandbyIsHot.java:188)
      
      1. HDFS-8772.01.patch
        0.9 kB
        Walter Su
      2. HDFS-8772.02.patch
        1 kB
        Walter Su
      3. HDFS-8772.03.patch
        4 kB
        Walter Su
      4. HDFS-8772.04.patch
        4 kB
        Walter Su
      5. HDFS-8772-branch-2.04.patch
        4 kB
        Walter Su

        Activity

        Hide
        walter.k.su Walter Su added a comment -

        causes
        cluster.waitActive() doesn't wait first BR to finish.

        How to reproduce quickly

        @@ -140,6 +151,7 @@ public void testDatanodeRestarts() throws Exception {
             HAUtil.setAllowStandbyReads(conf, true);
             conf.setLong(DFSConfigKeys.DFS_NAMENODE_ACCESSTIME_PRECISION_KEY, 0);
             conf.setInt(DFSConfigKeys.DFS_HA_TAILEDITS_PERIOD_KEY, 1);
        +    conf.setLong(DFS_BLOCKREPORT_INITIAL_DELAY_KEY, 200);
             MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
        
        Show
        walter.k.su Walter Su added a comment - causes cluster.waitActive() doesn't wait first BR to finish. How to reproduce quickly @@ -140,6 +151,7 @@ public void testDatanodeRestarts() throws Exception { HAUtil.setAllowStandbyReads(conf, true ); conf.setLong(DFSConfigKeys.DFS_NAMENODE_ACCESSTIME_PRECISION_KEY, 0); conf.setInt(DFSConfigKeys.DFS_HA_TAILEDITS_PERIOD_KEY, 1); + conf.setLong(DFS_BLOCKREPORT_INITIAL_DELAY_KEY, 200); MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Walter SuThanks for reporting.Nice work .. Patch,LGTM +1 ( non-binding)

        Show
        brahmareddy Brahma Reddy Battula added a comment - Walter Su Thanks for reporting.Nice work .. Patch,LGTM +1 ( non-binding)
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 5m 40s Findbugs (version ) appears to be broken on trunk.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 35s There were no new javac warning messages.
        +1 release audit 0m 19s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 33s There were no new checkstyle issues.
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 28s mvn install still works.
        +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
        +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 1m 7s Pre-build of native portion
        -1 hdfs tests 160m 28s Tests failed in hadoop-hdfs.
            180m 16s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestAppendSnapshotTruncate



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12745227/HDFS-8772.01.patch
        Optional Tests javac unit findbugs checkstyle
        git revision trunk / ac94ba3
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11697/artifact/patchprocess/whitespace.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11697/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11697/testReport/
        Java 1.7.0_55
        uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11697/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 5m 40s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 35s There were no new javac warning messages. +1 release audit 0m 19s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 33s There were no new checkstyle issues. -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 1m 7s Pre-build of native portion -1 hdfs tests 160m 28s Tests failed in hadoop-hdfs.     180m 16s   Reason Tests Failed unit tests hadoop.hdfs.TestAppendSnapshotTruncate Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12745227/HDFS-8772.01.patch Optional Tests javac unit findbugs checkstyle git revision trunk / ac94ba3 whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11697/artifact/patchprocess/whitespace.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11697/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11697/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11697/console This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        Hey Walter, the DN will send FBRs on startup right? It looks like the issue in the test is a race condition where the DN have registered but not FBR'd; could we wait for that startup FBR rather than manually triggering a new one? Seems more realistic.

        Show
        andrew.wang Andrew Wang added a comment - Hey Walter, the DN will send FBRs on startup right? It looks like the issue in the test is a race condition where the DN have registered but not FBR'd; could we wait for that startup FBR rather than manually triggering a new one? Seems more realistic.
        Hide
        walter.k.su Walter Su added a comment -

        Thanks Andrew Wang for review!
        >... could we wait for that startup FBR...
        That's also my first thought. But how much time should it wait? I think it should wait more than the interval of heartBeat.

        Uploaded 02 patch.

        Also I'm thinking add a helper function later, in MiniCluster to wait for first BR. sleep 200ms every test, just like waitActive() does. Because sleeping a rough number of time( for example, sleep 2000ms) is kind of wasting time.

        Show
        walter.k.su Walter Su added a comment - Thanks Andrew Wang for review! >... could we wait for that startup FBR... That's also my first thought. But how much time should it wait? I think it should wait more than the interval of heartBeat. Uploaded 02 patch. Also I'm thinking add a helper function later, in MiniCluster to wait for first BR. sleep 200ms every test, just like waitActive() does. Because sleeping a rough number of time( for example, sleep 2000ms) is kind of wasting time.
        Hide
        andrew.wang Andrew Wang added a comment -

        Hey Walter,

        We have this function FSNamesystem#getNumStaleStorages() which I think does what we need, see also DatanodeStorageInfo#areBlockContentsStale(). I think we could use GenericTestUtils#waitFor to wait for this to hit 0, waitFor is similar to your idea about polling every 200ms.

        Show
        andrew.wang Andrew Wang added a comment - Hey Walter, We have this function FSNamesystem#getNumStaleStorages() which I think does what we need, see also DatanodeStorageInfo#areBlockContentsStale(). I think we could use GenericTestUtils#waitFor to wait for this to hit 0, waitFor is similar to your idea about polling every 200ms.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 7m 36s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 35s There were no new javac warning messages.
        +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 21s There were no new checkstyle issues.
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 22s mvn install still works.
        +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
        +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 1m 6s Pre-build of native portion
        -1 hdfs tests 161m 37s Tests failed in hadoop-hdfs.
            184m 2s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestDistributedFileSystem



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12745712/HDFS-8772.02.patch
        Optional Tests javac unit findbugs checkstyle
        git revision trunk / d96bbe1
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11734/artifact/patchprocess/whitespace.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11734/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11734/testReport/
        Java 1.7.0_55
        uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11734/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 7m 36s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 35s There were no new javac warning messages. +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 21s There were no new checkstyle issues. -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 22s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 1m 6s Pre-build of native portion -1 hdfs tests 161m 37s Tests failed in hadoop-hdfs.     184m 2s   Reason Tests Failed unit tests hadoop.hdfs.TestDistributedFileSystem Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12745712/HDFS-8772.02.patch Optional Tests javac unit findbugs checkstyle git revision trunk / d96bbe1 whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11734/artifact/patchprocess/whitespace.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11734/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11734/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11734/console This message was automatically generated.
        Hide
        walter.k.su Walter Su added a comment -

        We have this function FSNamesystem#getNumStaleStorages() which I think does what we need, see also DatanodeStorageInfo#areBlockContentsStale(). I think we could use GenericTestUtils#waitFor to wait for this to hit 0, waitFor is similar to your idea about polling every 200ms.

        A quickly restarted DN doesn't affect them. GenericTestUtils is very helpful.
        I checked DatanodeDescriptor#updateRegInfo(DatanodeID). I think we can utilize blockReportCount.

        Uploaded 03 patch.

        Show
        walter.k.su Walter Su added a comment - We have this function FSNamesystem#getNumStaleStorages() which I think does what we need, see also DatanodeStorageInfo#areBlockContentsStale(). I think we could use GenericTestUtils#waitFor to wait for this to hit 0, waitFor is similar to your idea about polling every 200ms. A quickly restarted DN doesn't affect them. GenericTestUtils is very helpful. I checked DatanodeDescriptor#updateRegInfo(DatanodeID). I think we can utilize blockReportCount. Uploaded 03 patch.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 7m 47s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
        +1 javac 7m 32s There were no new javac warning messages.
        +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 25s There were no new checkstyle issues.
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 22s mvn install still works.
        +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
        +1 findbugs 2m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 1m 5s Pre-build of native portion
        -1 hdfs tests 161m 27s Tests failed in hadoop-hdfs.
            184m 3s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestDistributedFileSystem



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12745787/HDFS-8772.03.patch
        Optional Tests javac unit findbugs checkstyle
        git revision trunk / ee36f4f
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11738/artifact/patchprocess/whitespace.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11738/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11738/testReport/
        Java 1.7.0_55
        uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11738/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 7m 47s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 32s There were no new javac warning messages. +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 25s There were no new checkstyle issues. -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 22s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 2m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 1m 5s Pre-build of native portion -1 hdfs tests 161m 27s Tests failed in hadoop-hdfs.     184m 3s   Reason Tests Failed unit tests hadoop.hdfs.TestDistributedFileSystem Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12745787/HDFS-8772.03.patch Optional Tests javac unit findbugs checkstyle git revision trunk / ee36f4f whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11738/artifact/patchprocess/whitespace.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11738/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11738/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11738/console This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        LGTM, just one q, should we bump up the waitFor to be less aggressive like 30s, and the overall test timeout from 60s to 120s? Test boxes can be overloaded at times and run slowly, so I'd like some more safety to avoid flakiness.

        Show
        andrew.wang Andrew Wang added a comment - LGTM, just one q, should we bump up the waitFor to be less aggressive like 30s, and the overall test timeout from 60s to 120s? Test boxes can be overloaded at times and run slowly, so I'd like some more safety to avoid flakiness.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Actual testcase may cost 4.4 sec, if jenkin machine overloaded, i think, only we can bump the wait time to 10 sec is sufficient.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Actual testcase may cost 4.4 sec, if jenkin machine overloaded, i think, only we can bump the wait time to 10 sec is sufficient.
        Hide
        walter.k.su Walter Su added a comment -

        And I'm pretty sure waitFor 3s is enough. TestProcessCorruptBlocks even sleep 3s to wait full BR. I changed it to 10s to make sure, according to Brahma Reddy Battula's comment.

        Show
        walter.k.su Walter Su added a comment - And I'm pretty sure waitFor 3s is enough. TestProcessCorruptBlocks even sleep 3s to wait full BR. I changed it to 10s to make sure, according to Brahma Reddy Battula 's comment.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 7m 54s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
        +1 javac 7m 35s There were no new javac warning messages.
        +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 21s There were no new checkstyle issues.
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 19s mvn install still works.
        +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
        +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 1m 6s Pre-build of native portion
        +1 hdfs tests 161m 49s Tests passed in hadoop-hdfs.
            184m 34s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12748771/HDFS-8772.04.patch
        Optional Tests javac unit findbugs checkstyle
        git revision trunk / d540374
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11899/artifact/patchprocess/whitespace.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11899/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11899/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11899/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 7m 54s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 35s There were no new javac warning messages. +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 21s There were no new checkstyle issues. -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 19s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 1m 6s Pre-build of native portion +1 hdfs tests 161m 49s Tests passed in hadoop-hdfs.     184m 34s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12748771/HDFS-8772.04.patch Optional Tests javac unit findbugs checkstyle git revision trunk / d540374 whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11899/artifact/patchprocess/whitespace.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11899/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11899/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11899/console This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        Sure, the point of test timeouts is more so the build doesn't hang for hours, not to be particularly stringent about them. Anyways thanks for the work here all, I'll commit shortly.

        Show
        andrew.wang Andrew Wang added a comment - Sure, the point of test timeouts is more so the build doesn't hang for hours, not to be particularly stringent about them. Anyways thanks for the work here all, I'll commit shortly.
        Hide
        andrew.wang Andrew Wang added a comment -

        Ran into some issues backporting to branch-2, Walter Su mind preparing another patch for branch-2? We conflict a bit with HDFS-6440 in MiniDFSCluster, hoping we can fix by including some of those refactors.

        Show
        andrew.wang Andrew Wang added a comment - Ran into some issues backporting to branch-2, Walter Su mind preparing another patch for branch-2? We conflict a bit with HDFS-6440 in MiniDFSCluster, hoping we can fix by including some of those refactors.
        Hide
        walter.k.su Walter Su added a comment -

        Upload a patch for branch-2. Thanks Andrew Wang for take care of that.

        Show
        walter.k.su Walter Su added a comment - Upload a patch for branch-2. Thanks Andrew Wang for take care of that.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 7m 48s Pre-patch branch-2 compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
        +1 javac 5m 52s There were no new javac warning messages.
        +1 release audit 0m 19s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 23s There were no new checkstyle issues.
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 5s mvn install still works.
        +1 eclipse:eclipse 0m 30s The patch built with eclipse:eclipse.
        +1 findbugs 2m 27s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 1m 2s Pre-build of native portion
        -1 hdfs tests 170m 14s Tests failed in hadoop-hdfs.
            190m 46s  



        Reason Tests
        Failed unit tests hadoop.fs.TestWebHdfsFileContextMainOperations
          hadoop.hdfs.TestDistributedFileSystem
          hadoop.fs.TestSWebHdfsFileContextMainOperations
        Timed out tests org.apache.hadoop.cli.TestHDFSCLI



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12749193/HDFS-8772-branch-2.04.patch
        Optional Tests javac unit findbugs checkstyle
        git revision branch-2 / 42a05d2
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11932/artifact/patchprocess/whitespace.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11932/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11932/testReport/
        Java 1.7.0_55
        uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11932/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 7m 48s Pre-patch branch-2 compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 5m 52s There were no new javac warning messages. +1 release audit 0m 19s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 23s There were no new checkstyle issues. -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 5s mvn install still works. +1 eclipse:eclipse 0m 30s The patch built with eclipse:eclipse. +1 findbugs 2m 27s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 1m 2s Pre-build of native portion -1 hdfs tests 170m 14s Tests failed in hadoop-hdfs.     190m 46s   Reason Tests Failed unit tests hadoop.fs.TestWebHdfsFileContextMainOperations   hadoop.hdfs.TestDistributedFileSystem   hadoop.fs.TestSWebHdfsFileContextMainOperations Timed out tests org.apache.hadoop.cli.TestHDFSCLI Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12749193/HDFS-8772-branch-2.04.patch Optional Tests javac unit findbugs checkstyle git revision branch-2 / 42a05d2 whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11932/artifact/patchprocess/whitespace.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11932/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11932/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11932/console This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        Thanks Walter again for working on this, committed to trunk and branch-2.

        Show
        andrew.wang Andrew Wang added a comment - Thanks Walter again for working on this, committed to trunk and branch-2.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8278 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8278/)
        HDFS-8772. Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. (wang: rev 98a27d110129c7b32455035831480f1c6197260b)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyIsHot.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8278 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8278/ ) HDFS-8772 . Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. (wang: rev 98a27d110129c7b32455035831480f1c6197260b) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyIsHot.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #281 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/281/)
        HDFS-8772. Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. (wang: rev 98a27d110129c7b32455035831480f1c6197260b)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyIsHot.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #281 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/281/ ) HDFS-8772 . Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. (wang: rev 98a27d110129c7b32455035831480f1c6197260b) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyIsHot.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1011 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1011/)
        HDFS-8772. Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. (wang: rev 98a27d110129c7b32455035831480f1c6197260b)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyIsHot.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1011 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1011/ ) HDFS-8772 . Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. (wang: rev 98a27d110129c7b32455035831480f1c6197260b) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyIsHot.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2208 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2208/)
        HDFS-8772. Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. (wang: rev 98a27d110129c7b32455035831480f1c6197260b)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyIsHot.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2208 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2208/ ) HDFS-8772 . Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. (wang: rev 98a27d110129c7b32455035831480f1c6197260b) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyIsHot.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #270 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/270/)
        HDFS-8772. Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. (wang: rev 98a27d110129c7b32455035831480f1c6197260b)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyIsHot.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #270 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/270/ ) HDFS-8772 . Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. (wang: rev 98a27d110129c7b32455035831480f1c6197260b) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyIsHot.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #278 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/278/)
        HDFS-8772. Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. (wang: rev 98a27d110129c7b32455035831480f1c6197260b)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyIsHot.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #278 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/278/ ) HDFS-8772 . Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. (wang: rev 98a27d110129c7b32455035831480f1c6197260b) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyIsHot.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2227 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2227/)
        HDFS-8772. Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. (wang: rev 98a27d110129c7b32455035831480f1c6197260b)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyIsHot.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2227 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2227/ ) HDFS-8772 . Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. (wang: rev 98a27d110129c7b32455035831480f1c6197260b) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyIsHot.java
        Hide
        kihwal Kihwal Lee added a comment -

        Cherry-picked to branch-2.7 as this test case occasionally fails the same way in branch-2.7.

        Show
        kihwal Kihwal Lee added a comment - Cherry-picked to branch-2.7 as this test case occasionally fails the same way in branch-2.7.
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        Closing the JIRA as part of 2.7.3 release.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.

          People

          • Assignee:
            walter.k.su Walter Su
            Reporter:
            walter.k.su Walter Su
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development