Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6101

TestReplaceDatanodeOnFailure fails occasionally

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Exception details in a comment below.

      The failure repros on both OS X and Linux if I run the test ~10 times in a loop.

      1. HDFS-6101.001.patch
        5 kB
        Wei-Chiu Chuang
      2. HDFS-6101.002.patch
        5 kB
        Wei-Chiu Chuang
      3. HDFS-6101.003.patch
        5 kB
        Wei-Chiu Chuang
      4. HDFS-6101.004.patch
        6 kB
        Wei-Chiu Chuang
      5. HDFS-6101.005.patch
        5 kB
        Wei-Chiu Chuang
      6. HDFS-6101.006.patch
        6 kB
        Wei-Chiu Chuang
      7. HDFS-6101.007.patch
        5 kB
        Wei-Chiu Chuang
      8. TestReplaceDatanodeOnFailure.log
        846 kB
        Arpit Agarwal

        Issue Links

          Activity

          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Full log from test run attached.

          Show
          arpitagarwal Arpit Agarwal added a comment - Full log from test run attached.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Exception details:

          testReplaceDatanodeOnFailure(org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure)  Time elapsed: 25.176 sec  <<< FAILURE!java.lang.AssertionError: expected:<3> but was:<2>  at org.junit.Assert.fail(Assert.java:93)
            at org.junit.Assert.failNotEquals(Assert.java:647)  at org.junit.Assert.assertEquals(Assert.java:128)
            at org.junit.Assert.assertEquals(Assert.java:472)  at org.junit.Assert.assertEquals(Assert.java:456)
            at org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure$SlowWriter.checkReplication(TestReplaceDatanodeOnFailure.java:234)
            at org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure.testReplaceDatanodeOnFailure(TestReplaceDatanodeOnFailure.java:153)
          
          Show
          arpitagarwal Arpit Agarwal added a comment - Exception details: testReplaceDatanodeOnFailure(org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure) Time elapsed: 25.176 sec <<< FAILURE!java.lang.AssertionError: expected:<3> but was:<2> at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.junit.Assert.assertEquals(Assert.java:456) at org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure$SlowWriter.checkReplication(TestReplaceDatanodeOnFailure.java:234) at org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure.testReplaceDatanodeOnFailure(TestReplaceDatanodeOnFailure.java:153)
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Thanks for reporting this, Arpit Agarwal,
          I was able to reproduce the bug, and am working on resolving this issue. Assigning this JIRA to myself.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Thanks for reporting this, Arpit Agarwal , I was able to reproduce the bug, and am working on resolving this issue. Assigning this JIRA to myself.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Thanks for picking this up Wei-Chiu Chuang.

          Show
          arpitagarwal Arpit Agarwal added a comment - Thanks for picking this up Wei-Chiu Chuang .
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          patch rev 01: eliminated some unsynchronized access to shared variables, and make sure data nodes are active before writing files.

          Show
          jojochuang Wei-Chiu Chuang added a comment - patch rev 01: eliminated some unsynchronized access to shared variables, and make sure data nodes are active before writing files.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          For reference, before the patch, the test case failed 16 out of 100 times on my local machine. After the patch, the test case failed none.

          Show
          jojochuang Wei-Chiu Chuang added a comment - For reference, before the patch, the test case failed 16 out of 100 times on my local machine. After the patch, the test case failed none.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 7m 51s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 8m 1s There were no new javac warning messages.
          +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 24s There were no new checkstyle issues.
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 28s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 1m 2s Pre-build of native portion
          -1 hdfs tests 51m 25s Tests failed in hadoop-hdfs.
              74m 38s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.datanode.TestFsDatasetCache
            hadoop.hdfs.web.TestWebHDFS
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
            hadoop.hdfs.TestRollingUpgrade
            hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12766437/HDFS-6101.001.patch
          Optional Tests javac unit findbugs checkstyle
          git revision trunk / 40cac59
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12966/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12966/artifact/patchprocess/whitespace.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12966/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12966/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12966/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 7m 51s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 1s There were no new javac warning messages. +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 24s There were no new checkstyle issues. -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 1m 2s Pre-build of native portion -1 hdfs tests 51m 25s Tests failed in hadoop-hdfs.     74m 38s   Reason Tests Failed unit tests hadoop.hdfs.server.datanode.TestFsDatasetCache   hadoop.hdfs.web.TestWebHDFS   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration   hadoop.hdfs.TestRollingUpgrade   hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12766437/HDFS-6101.001.patch Optional Tests javac unit findbugs checkstyle git revision trunk / 40cac59 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12966/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12966/artifact/patchprocess/whitespace.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12966/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12966/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12966/console This message was automatically generated.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Rev2 removes the empty line with space. This patch only fixes the flaky test, and therefore the failed tests are unrelated.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Rev2 removes the empty line with space. This patch only fixes the flaky test, and therefore the failed tests are unrelated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 9m 5s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 9m 10s There were no new javac warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 38s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 39s mvn install still works.
          +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse.
          +1 findbugs 2m 53s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 1m 14s Pre-build of native portion
          -1 hdfs tests 56m 8s Tests failed in hadoop-hdfs.
              82m 51s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.blockmanagement.TestNodeCount
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure000
            hadoop.hdfs.TestLeaseRecovery2



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12766457/HDFS-6101.002.patch
          Optional Tests javac unit findbugs checkstyle
          git revision trunk / 40cac59
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12969/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12969/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12969/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12969/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 9m 5s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 9m 10s There were no new javac warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 38s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 39s mvn install still works. +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse. +1 findbugs 2m 53s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 1m 14s Pre-build of native portion -1 hdfs tests 56m 8s Tests failed in hadoop-hdfs.     82m 51s   Reason Tests Failed unit tests hadoop.hdfs.server.blockmanagement.TestNodeCount   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure000   hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12766457/HDFS-6101.002.patch Optional Tests javac unit findbugs checkstyle git revision trunk / 40cac59 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12969/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12969/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12969/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12969/console This message was automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Thanks Wei-Chiu Chuang. I can't repro the failure anymore but your fix looks valid.

          Can you fix the coding style issues? e.g. Space before } in {{synchronized (this){, remove extra spaces in synchronized ( writer ) etc.?

          Show
          arpitagarwal Arpit Agarwal added a comment - Thanks Wei-Chiu Chuang . I can't repro the failure anymore but your fix looks valid. Can you fix the coding style issues? e.g. Space before } in {{synchronized (this){ , remove extra spaces in synchronized ( writer ) etc.?
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Thanks Arpit Agarwal for the review. Here's the code with updated code style. Kind of surprised the tool did not pick up these white space issues.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Thanks Arpit Agarwal for the review. Here's the code with updated code style. Kind of surprised the tool did not pick up these white space issues.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 10m 43s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 10m 36s There were no new javac warning messages.
          -1 release audit 0m 21s The applied patch generated 1 release audit warnings.
          +1 checkstyle 1m 58s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 57s mvn install still works.
          +1 eclipse:eclipse 0m 43s The patch built with eclipse:eclipse.
          +1 findbugs 3m 19s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 1m 21s Pre-build of native portion
          -1 hdfs tests 63m 30s Tests failed in hadoop-hdfs.
              94m 32s  



          Reason Tests
          Failed unit tests hadoop.fs.TestGlobPaths
            hadoop.hdfs.server.namenode.TestFileContextAcl
            hadoop.hdfs.TestReplaceDatanodeOnFailure
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer
          Timed out tests org.apache.hadoop.hdfs.server.balancer.TestBalancer
            org.apache.hadoop.hdfs.server.blockmanagement.TestPendingReplication



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12766672/HDFS-6101.003.patch
          Optional Tests javac unit findbugs checkstyle
          git revision trunk / be7a0ad
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12993/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12993/artifact/patchprocess/patchReleaseAuditProblems.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12993/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12993/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12993/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 10m 43s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 10m 36s There were no new javac warning messages. -1 release audit 0m 21s The applied patch generated 1 release audit warnings. +1 checkstyle 1m 58s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 57s mvn install still works. +1 eclipse:eclipse 0m 43s The patch built with eclipse:eclipse. +1 findbugs 3m 19s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 1m 21s Pre-build of native portion -1 hdfs tests 63m 30s Tests failed in hadoop-hdfs.     94m 32s   Reason Tests Failed unit tests hadoop.fs.TestGlobPaths   hadoop.hdfs.server.namenode.TestFileContextAcl   hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.server.namenode.ha.TestEditLogTailer Timed out tests org.apache.hadoop.hdfs.server.balancer.TestBalancer   org.apache.hadoop.hdfs.server.blockmanagement.TestPendingReplication Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12766672/HDFS-6101.003.patch Optional Tests javac unit findbugs checkstyle git revision trunk / be7a0ad Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12993/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12993/artifact/patchprocess/patchReleaseAuditProblems.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12993/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12993/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12993/console This message was automatically generated.
          Hide
          walter.k.su Walter Su added a comment -

          The test failed is possibly because the stopped DN doesn't be removed from cluster map, and sleepSeconds(5) doesn't make sure it's removed from cluster map.

          1. Please don't remove this. It's intended. After sleeping, we want some writer NOT yet started.

          -      // Some of them are too slow and will be not yet started. 
          -      sleepSeconds(1);
          

          2. Instead of hardcode sleep time 5s. We can use GenericTestUtils.waitFor(..) to check the block replication. The wait/notify is unnecessary.

          3. After

          cluster.stopDataNode(AppendTestUtil.nextInt(REPLICATION));
          

          We should call cluster.setDataNodeDead(..) to remove it from cluster map.

          Show
          walter.k.su Walter Su added a comment - The test failed is possibly because the stopped DN doesn't be removed from cluster map, and sleepSeconds(5) doesn't make sure it's removed from cluster map. 1. Please don't remove this. It's intended. After sleeping, we want some writer NOT yet started. - // Some of them are too slow and will be not yet started. - sleepSeconds(1); 2. Instead of hardcode sleep time 5s. We can use GenericTestUtils.waitFor(..) to check the block replication. The wait/notify is unnecessary. 3. After cluster.stopDataNode(AppendTestUtil.nextInt(REPLICATION)); We should call cluster.setDataNodeDead(..) to remove it from cluster map.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Hi Walter Su Thanks for your comments!
          I am sorry I was swamped by other tasks and wasn't able to get up to this. I am looking to make a new revision based on your suggestions.

          Thanks.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Hi Walter Su Thanks for your comments! I am sorry I was swamped by other tasks and wasn't able to get up to this. I am looking to make a new revision based on your suggestions. Thanks.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Rev4 is based on Walter Su's comments. I tested it locally and it did not fail in 100 runs.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Rev4 is based on Walter Su 's comments. I tested it locally and it did not fail in 100 runs.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 5s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 3m 4s trunk passed
          +1 compile 0m 37s trunk passed with JDK v1.8.0_60
          +1 compile 0m 34s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 17s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          -1 findbugs 1m 58s hadoop-hdfs-project/hadoop-hdfs in trunk has 1 extant Findbugs warnings.
          +1 javadoc 1m 12s trunk passed with JDK v1.8.0_60
          +1 javadoc 1m 55s trunk passed with JDK v1.7.0_79
          +1 mvninstall 0m 43s the patch passed
          +1 compile 0m 36s the patch passed with JDK v1.8.0_60
          +1 javac 0m 36s the patch passed
          +1 compile 0m 34s the patch passed with JDK v1.7.0_79
          +1 javac 0m 34s the patch passed
          +1 checkstyle 0m 17s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 11s the patch passed
          +1 javadoc 1m 13s the patch passed with JDK v1.8.0_60
          +1 javadoc 1m 55s the patch passed with JDK v1.7.0_79
          -1 unit 57m 46s hadoop-hdfs in the patch failed with JDK v1.8.0_60.
          -1 unit 53m 49s hadoop-hdfs in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 20s Patch generated 58 ASF License warnings.
          132m 10s



          Reason Tests
          JDK v1.8.0_60 Failed junit tests hadoop.hdfs.server.blockmanagement.TestNodeCount
            hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
            hadoop.hdfs.server.datanode.TestDataNodeMetrics
            hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.server.namenode.ha.TestDNFencing
          JDK v1.7.0_79 Failed junit tests hadoop.hdfs.TestDatanodeRegistration
            hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica
            hadoop.hdfs.TestHDFSFileSystemContract



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-09
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771371/HDFS-6101.004.patch
          JIRA Issue HDFS-6101
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux c789c286e7c3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
          git revision trunk / ef926b2
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13440/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13440/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13440/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13440/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13440/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13440/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13440/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 227MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13440/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 5s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 3m 4s trunk passed +1 compile 0m 37s trunk passed with JDK v1.8.0_60 +1 compile 0m 34s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 17s trunk passed +1 mvneclipse 0m 14s trunk passed -1 findbugs 1m 58s hadoop-hdfs-project/hadoop-hdfs in trunk has 1 extant Findbugs warnings. +1 javadoc 1m 12s trunk passed with JDK v1.8.0_60 +1 javadoc 1m 55s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 43s the patch passed +1 compile 0m 36s the patch passed with JDK v1.8.0_60 +1 javac 0m 36s the patch passed +1 compile 0m 34s the patch passed with JDK v1.7.0_79 +1 javac 0m 34s the patch passed +1 checkstyle 0m 17s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 11s the patch passed +1 javadoc 1m 13s the patch passed with JDK v1.8.0_60 +1 javadoc 1m 55s the patch passed with JDK v1.7.0_79 -1 unit 57m 46s hadoop-hdfs in the patch failed with JDK v1.8.0_60. -1 unit 53m 49s hadoop-hdfs in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 20s Patch generated 58 ASF License warnings. 132m 10s Reason Tests JDK v1.8.0_60 Failed junit tests hadoop.hdfs.server.blockmanagement.TestNodeCount   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.server.datanode.TestDataNodeMetrics   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.namenode.ha.TestDNFencing JDK v1.7.0_79 Failed junit tests hadoop.hdfs.TestDatanodeRegistration   hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica   hadoop.hdfs.TestHDFSFileSystemContract Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-09 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771371/HDFS-6101.004.patch JIRA Issue HDFS-6101 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux c789c286e7c3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision trunk / ef926b2 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13440/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/13440/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13440/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13440/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13440/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13440/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13440/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 227MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13440/console This message was automatically generated.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          The findbugs warning is being resolved in HDFS-9401 and unrelated to this patch.
          Other test case failures are not related to this patch, because this patch only modified test code.

          Show
          jojochuang Wei-Chiu Chuang added a comment - The findbugs warning is being resolved in HDFS-9401 and unrelated to this patch. Other test case failures are not related to this patch, because this patch only modified test code.
          Hide
          cnauroth Chris Nauroth added a comment -

          Hi Wei-Chiu Chuang. This looks right to me. I just have a minor nitpick. Would you please remove this line instead of commenting it out?

                //sleepSeconds(5);
          

          After that, I think the patch will be ready to go.

          Show
          cnauroth Chris Nauroth added a comment - Hi Wei-Chiu Chuang . This looks right to me. I just have a minor nitpick. Would you please remove this line instead of commenting it out? //sleepSeconds(5); After that, I think the patch will be ready to go.
          Hide
          walter.k.su Walter Su added a comment -

          Sorry I almost forgot this. I think synchronized keyword also can be removed?

          Show
          walter.k.su Walter Su added a comment - Sorry I almost forgot this. I think synchronized keyword also can be removed?
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Thanks for the comments and reviews.
          I was hesitate to continue on it because on my local machine, this test still frequently failed after I made the change suggested by Walter Su.

          The only way I could avoid failures is to reduce the number of concurrent writers. Originally the test had 10 writers, and it fails 1 in 4 times. I reduced the number to 5, and it did not fail for 100 runs. This is why I suspect there is another issue with the default block placement policy (HDFS-9361 Default block placement policy causes TestReplaceDataNodeOnFailure to fail intermittently) When there are 10 writers begin to writer at the same time, the policy will not allow some writers set up pipelines with 3 data nodes, due to the load factor of data nodes. When the writers reduces, the load reduces and therefore the test passed.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Thanks for the comments and reviews. I was hesitate to continue on it because on my local machine, this test still frequently failed after I made the change suggested by Walter Su . The only way I could avoid failures is to reduce the number of concurrent writers. Originally the test had 10 writers, and it fails 1 in 4 times. I reduced the number to 5, and it did not fail for 100 runs. This is why I suspect there is another issue with the default block placement policy ( HDFS-9361 Default block placement policy causes TestReplaceDataNodeOnFailure to fail intermittently) When there are 10 writers begin to writer at the same time, the policy will not allow some writers set up pipelines with 3 data nodes, due to the load factor of data nodes. When the writers reduces, the load reduces and therefore the test passed.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          I am attach rev5, which reduced the number of writers to 5.

          Show
          jojochuang Wei-Chiu Chuang added a comment - I am attach rev5, which reduced the number of writers to 5.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 9m 46s trunk passed
          +1 compile 0m 58s trunk passed with JDK v1.8.0_66
          +1 compile 0m 51s trunk passed with JDK v1.7.0_85
          +1 checkstyle 0m 19s trunk passed
          +1 mvnsite 1m 6s trunk passed
          +1 mvneclipse 0m 17s trunk passed
          +1 findbugs 2m 22s trunk passed
          +1 javadoc 1m 22s trunk passed with JDK v1.8.0_66
          +1 javadoc 2m 7s trunk passed with JDK v1.7.0_85
          +1 mvninstall 0m 59s the patch passed
          +1 compile 0m 56s the patch passed with JDK v1.8.0_66
          +1 javac 0m 56s the patch passed
          +1 compile 0m 53s the patch passed with JDK v1.7.0_85
          +1 javac 0m 53s the patch passed
          +1 checkstyle 0m 19s the patch passed
          +1 mvnsite 1m 6s the patch passed
          +1 mvneclipse 0m 18s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 34s the patch passed
          +1 javadoc 1m 21s the patch passed with JDK v1.8.0_66
          +1 javadoc 2m 4s the patch passed with JDK v1.7.0_85
          -1 unit 64m 18s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          -1 unit 57m 52s hadoop-hdfs in the patch failed with JDK v1.7.0_85.
          -1 asflicense 0m 21s Patch generated 56 ASF License warnings.
          155m 29s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
            hadoop.hdfs.TestDFSStripedOutputStream
            hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
            hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
          JDK v1.7.0_85 Failed junit tests hadoop.hdfs.TestRecoverStripedFile



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:date2015-11-20
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12773584/HDFS-6101.005.patch
          JIRA Issue HDFS-6101
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux db70bb425eda 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-3f4279a/precommit/personality/hadoop.sh
          git revision trunk / 4539131
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13585/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13585/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13585/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13585/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13585/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13585/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 77MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13585/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 46s trunk passed +1 compile 0m 58s trunk passed with JDK v1.8.0_66 +1 compile 0m 51s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 19s trunk passed +1 mvnsite 1m 6s trunk passed +1 mvneclipse 0m 17s trunk passed +1 findbugs 2m 22s trunk passed +1 javadoc 1m 22s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 7s trunk passed with JDK v1.7.0_85 +1 mvninstall 0m 59s the patch passed +1 compile 0m 56s the patch passed with JDK v1.8.0_66 +1 javac 0m 56s the patch passed +1 compile 0m 53s the patch passed with JDK v1.7.0_85 +1 javac 0m 53s the patch passed +1 checkstyle 0m 19s the patch passed +1 mvnsite 1m 6s the patch passed +1 mvneclipse 0m 18s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 34s the patch passed +1 javadoc 1m 21s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 4s the patch passed with JDK v1.7.0_85 -1 unit 64m 18s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 57m 52s hadoop-hdfs in the patch failed with JDK v1.7.0_85. -1 asflicense 0m 21s Patch generated 56 ASF License warnings. 155m 29s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   hadoop.hdfs.TestDFSStripedOutputStream   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits JDK v1.7.0_85 Failed junit tests hadoop.hdfs.TestRecoverStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:date2015-11-20 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12773584/HDFS-6101.005.patch JIRA Issue HDFS-6101 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux db70bb425eda 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-3f4279a/precommit/personality/hadoop.sh git revision trunk / 4539131 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/13585/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13585/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13585/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13585/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13585/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13585/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 77MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13585/console This message was automatically generated.
          Hide
          walter.k.su Walter Su added a comment -

          When there are 10 writers begin to writer at the same time, the policy will not allow some writers set up pipelines with 3 data nodes, due to the load factor of data nodes.

          It happens because in test we only start few DNs and write a lot files. In production It won't be a problem. I saw nodes be excluded by placement policy many times when I write tests for erasue-coded files, which writes to 9 DNs concurrently only for one file.

          So could you try
          conf.setBoolean(DFSConfigKeys.DFS_NAMENODE_REPLICATION_CONSIDERLOAD_KEY, false);
          and don't reduce the writers?

          Show
          walter.k.su Walter Su added a comment - When there are 10 writers begin to writer at the same time, the policy will not allow some writers set up pipelines with 3 data nodes, due to the load factor of data nodes. It happens because in test we only start few DNs and write a lot files. In production It won't be a problem. I saw nodes be excluded by placement policy many times when I write tests for erasue-coded files, which writes to 9 DNs concurrently only for one file. So could you try conf.setBoolean(DFSConfigKeys.DFS_NAMENODE_REPLICATION_CONSIDERLOAD_KEY, false); and don't reduce the writers?
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Thanks for the suggestion! That should resolve the issue. Let me try that later.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Thanks for the suggestion! That should resolve the issue. Let me try that later.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Attaching rev6.

          In this version, ignore load factor when choosing data node pipelines. In addition, sleep 3 seconds instead of 1 seconds, to let clients write something before adding/removing data nodes. (This is necessary, otherwise some clients will still choose 2 data nodes instead of 3)

          Show
          jojochuang Wei-Chiu Chuang added a comment - Attaching rev6. In this version, ignore load factor when choosing data node pipelines. In addition, sleep 3 seconds instead of 1 seconds, to let clients write something before adding/removing data nodes. (This is necessary, otherwise some clients will still choose 2 data nodes instead of 3)
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          I am currently running rev6 locally and so far nothing failed in more than 100 runs.

          Show
          jojochuang Wei-Chiu Chuang added a comment - I am currently running rev6 locally and so far nothing failed in more than 100 runs.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 10m 16s trunk passed
          +1 compile 1m 8s trunk passed with JDK v1.8.0_66
          +1 compile 0m 57s trunk passed with JDK v1.7.0_85
          +1 checkstyle 0m 21s trunk passed
          +1 mvnsite 1m 10s trunk passed
          +1 mvneclipse 0m 18s trunk passed
          +1 findbugs 2m 29s trunk passed
          +1 javadoc 1m 34s trunk passed with JDK v1.8.0_66
          +1 javadoc 2m 34s trunk passed with JDK v1.7.0_85
          +1 mvninstall 1m 10s the patch passed
          +1 compile 1m 9s the patch passed with JDK v1.8.0_66
          +1 javac 1m 9s the patch passed
          +1 compile 0m 59s the patch passed with JDK v1.7.0_85
          +1 javac 0m 59s the patch passed
          +1 checkstyle 0m 20s the patch passed
          +1 mvnsite 1m 14s the patch passed
          +1 mvneclipse 0m 16s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 46s the patch passed
          +1 javadoc 1m 34s the patch passed with JDK v1.8.0_66
          +1 javadoc 2m 26s the patch passed with JDK v1.7.0_85
          -1 unit 86m 53s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          -1 unit 81m 36s hadoop-hdfs in the patch failed with JDK v1.7.0_85.
          -1 asflicense 0m 25s Patch generated 56 ASF License warnings.
          205m 12s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.TestDFSClientExcludedNodes
            hadoop.hdfs.TestWriteReadStripedFile
            hadoop.hdfs.server.datanode.TestBlockReplacement
            hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
            hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
            hadoop.hdfs.TestEncryptionZones
            hadoop.security.TestPermission
            hadoop.hdfs.server.datanode.TestDirectoryScanner
          JDK v1.7.0_85 Failed junit tests hadoop.hdfs.TestDFSUpgradeFromImage
            hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
            hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
            hadoop.security.TestPermission
            hadoop.hdfs.server.datanode.TestDirectoryScanner



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12773855/HDFS-6101.006.patch
          JIRA Issue HDFS-6101
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 16e57ecd8bb4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 201f14e
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13619/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13619/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13619/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13619/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13619/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13619/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 75MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13619/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 10m 16s trunk passed +1 compile 1m 8s trunk passed with JDK v1.8.0_66 +1 compile 0m 57s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 21s trunk passed +1 mvnsite 1m 10s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 2m 29s trunk passed +1 javadoc 1m 34s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 34s trunk passed with JDK v1.7.0_85 +1 mvninstall 1m 10s the patch passed +1 compile 1m 9s the patch passed with JDK v1.8.0_66 +1 javac 1m 9s the patch passed +1 compile 0m 59s the patch passed with JDK v1.7.0_85 +1 javac 0m 59s the patch passed +1 checkstyle 0m 20s the patch passed +1 mvnsite 1m 14s the patch passed +1 mvneclipse 0m 16s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 46s the patch passed +1 javadoc 1m 34s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 26s the patch passed with JDK v1.7.0_85 -1 unit 86m 53s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 81m 36s hadoop-hdfs in the patch failed with JDK v1.7.0_85. -1 asflicense 0m 25s Patch generated 56 ASF License warnings. 205m 12s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.TestDFSClientExcludedNodes   hadoop.hdfs.TestWriteReadStripedFile   hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.hdfs.TestEncryptionZones   hadoop.security.TestPermission   hadoop.hdfs.server.datanode.TestDirectoryScanner JDK v1.7.0_85 Failed junit tests hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   hadoop.hdfs.server.namenode.TestFSImageWithSnapshot   hadoop.security.TestPermission   hadoop.hdfs.server.datanode.TestDirectoryScanner Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12773855/HDFS-6101.006.patch JIRA Issue HDFS-6101 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 16e57ecd8bb4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 201f14e findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/13619/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13619/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13619/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13619/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13619/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13619/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 75MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13619/console This message was automatically generated.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          None of the test failures is attributed to this patch, because this patch only fixed test code.

          Hi Walter Su could you review again? Thank you.

          Show
          jojochuang Wei-Chiu Chuang added a comment - None of the test failures is attributed to this patch, because this patch only fixed test code. Hi Walter Su could you review again? Thank you.
          Hide
          walter.k.su Walter Su added a comment -

          Thanks for the update.

          We should call cluster.setDataNodeDead(..) to remove it from cluster map.

          1. Actually it's wrong. My mistake. This line is unnecessary.

          2. Suggestion, you can enable log to make debug easier.

          static {
            ((Log4JLogger)LogFactory.getLog(BlockPlacementPolicy.class))
                .getLogger().setLevel(Level.ALL);
          }
          

          3.

          sleep 3 seconds instead of 1 seconds

          It's not exactly the intention of the old logic. I tried sleep(1), and I found

          org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy$NotEnoughReplicasException: [
          Node /rack1/127.0.0.1:44392 [
            Datanode 127.0.0.1:44392 is not chosen since no good storage to place the block .
          ]
          

          That's because the first block report is not finished, so DatanodeDescriptor#storageMap is empty. I tried cluster.waitFirstBRCompleted(); but there is race condition with the slowwriters.

          So I think we can:
          1. start 5 writers, and sleep shortly to make them all started.
          2. start 2 new DNs, waitFirstBRCompleted, and stop an old DN. (We don't need to call cluster.setDataNodeDead())
          3. start 5 new writers.

          As the comment says

                // Let slow writers write something.
                // Some of them are too slow and will be not yet started. 
          

          In this way, we don't change the logic of the test.

          4. This line is not needed.

          final BlockManager bm =
          
          Show
          walter.k.su Walter Su added a comment - Thanks for the update. We should call cluster.setDataNodeDead(..) to remove it from cluster map. 1. Actually it's wrong. My mistake. This line is unnecessary. 2. Suggestion, you can enable log to make debug easier. static { ((Log4JLogger)LogFactory.getLog(BlockPlacementPolicy.class)) .getLogger().setLevel(Level.ALL); } 3. sleep 3 seconds instead of 1 seconds It's not exactly the intention of the old logic. I tried sleep(1), and I found org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy$NotEnoughReplicasException: [ Node /rack1/127.0.0.1:44392 [ Datanode 127.0.0.1:44392 is not chosen since no good storage to place the block . ] That's because the first block report is not finished, so DatanodeDescriptor#storageMap is empty. I tried cluster.waitFirstBRCompleted(); but there is race condition with the slowwriters . So I think we can: 1. start 5 writers, and sleep shortly to make them all started. 2. start 2 new DNs, waitFirstBRCompleted, and stop an old DN. (We don't need to call cluster.setDataNodeDead()) 3. start 5 new writers. As the comment says // Let slow writers write something. // Some of them are too slow and will be not yet started. In this way, we don't change the logic of the test. 4. This line is not needed. final BlockManager bm =
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Thanks Walter Su for the detailed review. I think the points are reasonable, and am attaching rev07.

          I am running this version locally and so far so good. No failed tests in 100 runs.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Thanks Walter Su for the detailed review. I think the points are reasonable, and am attaching rev07. I am running this version locally and so far so good. No failed tests in 100 runs.
          Hide
          walter.k.su Walter Su added a comment -

          LGTM. +1 pending jenkins.

          Show
          walter.k.su Walter Su added a comment - LGTM. +1 pending jenkins.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 53s trunk passed
          +1 compile 0m 43s trunk passed with JDK v1.8.0_66
          +1 compile 0m 44s trunk passed with JDK v1.7.0_85
          +1 checkstyle 0m 19s trunk passed
          +1 mvnsite 0m 54s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 52s trunk passed
          +1 javadoc 1m 6s trunk passed with JDK v1.8.0_66
          +1 javadoc 1m 46s trunk passed with JDK v1.7.0_85
          +1 mvninstall 0m 48s the patch passed
          +1 compile 0m 41s the patch passed with JDK v1.8.0_66
          +1 javac 0m 41s the patch passed
          +1 compile 0m 39s the patch passed with JDK v1.7.0_85
          +1 javac 0m 39s the patch passed
          +1 checkstyle 0m 15s the patch passed
          +1 mvnsite 0m 53s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 2s the patch passed
          +1 javadoc 1m 8s the patch passed with JDK v1.8.0_66
          +1 javadoc 1m 47s the patch passed with JDK v1.7.0_85
          -1 unit 51m 18s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          -1 unit 50m 42s hadoop-hdfs in the patch failed with JDK v1.7.0_85.
          -1 asflicense 0m 20s Patch generated 58 ASF License warnings.
          128m 59s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
            hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
            hadoop.security.TestPermission
          JDK v1.7.0_85 Failed junit tests hadoop.security.TestPermission
            hadoop.hdfs.server.namenode.TestNameNodeMetricsLogger
            hadoop.hdfs.server.namenode.TestBackupNode



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12774020/HDFS-6101.007.patch
          JIRA Issue HDFS-6101
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 998c0c825f94 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / fb0f09e
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13632/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13632/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13632/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13632/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13632/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13632/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 75MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13632/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 53s trunk passed +1 compile 0m 43s trunk passed with JDK v1.8.0_66 +1 compile 0m 44s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 19s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 52s trunk passed +1 javadoc 1m 6s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 46s trunk passed with JDK v1.7.0_85 +1 mvninstall 0m 48s the patch passed +1 compile 0m 41s the patch passed with JDK v1.8.0_66 +1 javac 0m 41s the patch passed +1 compile 0m 39s the patch passed with JDK v1.7.0_85 +1 javac 0m 39s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 2s the patch passed +1 javadoc 1m 8s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 47s the patch passed with JDK v1.7.0_85 -1 unit 51m 18s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 50m 42s hadoop-hdfs in the patch failed with JDK v1.7.0_85. -1 asflicense 0m 20s Patch generated 58 ASF License warnings. 128m 59s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.security.TestPermission JDK v1.7.0_85 Failed junit tests hadoop.security.TestPermission   hadoop.hdfs.server.namenode.TestNameNodeMetricsLogger   hadoop.hdfs.server.namenode.TestBackupNode Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12774020/HDFS-6101.007.patch JIRA Issue HDFS-6101 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 998c0c825f94 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / fb0f09e findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/13632/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13632/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13632/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13632/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13632/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13632/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 75MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13632/console This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          +1 from me too. I verified that the test failures are unrelated and tracked elsewhere.

          Show
          cnauroth Chris Nauroth added a comment - +1 from me too. I verified that the test failures are unrelated and tracked elsewhere.
          Hide
          cnauroth Chris Nauroth added a comment -

          I have committed this to trunk and branch-2. Wei-Chiu Chuang, thank you for the patch. Walter Su and Arpit Agarwal, thank you for the code reviews.

          Show
          cnauroth Chris Nauroth added a comment - I have committed this to trunk and branch-2. Wei-Chiu Chuang , thank you for the patch. Walter Su and Arpit Agarwal , thank you for the code reviews.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Thanks Chris Nauroth and Walter Su for reviews, especially Walter assisted a lot in offering the best solution, and Arpit Agarwal for reporting the issue!

          Show
          jojochuang Wei-Chiu Chuang added a comment - Thanks Chris Nauroth and Walter Su for reviews, especially Walter assisted a lot in offering the best solution, and Arpit Agarwal for reporting the issue!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8875 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8875/)
          HDFS-6101. TestReplaceDatanodeOnFailure fails occasionally. Contributed (cnauroth: rev 1777608fa075a807c645619fda87cb8de1b0350c)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplaceDatanodeOnFailure.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8875 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8875/ ) HDFS-6101 . TestReplaceDatanodeOnFailure fails occasionally. Contributed (cnauroth: rev 1777608fa075a807c645619fda87cb8de1b0350c) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplaceDatanodeOnFailure.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1447 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1447/)
          HDFS-6101. TestReplaceDatanodeOnFailure fails occasionally. Contributed (cnauroth: rev 1777608fa075a807c645619fda87cb8de1b0350c)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplaceDatanodeOnFailure.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1447 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1447/ ) HDFS-6101 . TestReplaceDatanodeOnFailure fails occasionally. Contributed (cnauroth: rev 1777608fa075a807c645619fda87cb8de1b0350c) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplaceDatanodeOnFailure.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #714 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/714/)
          HDFS-6101. TestReplaceDatanodeOnFailure fails occasionally. Contributed (cnauroth: rev 1777608fa075a807c645619fda87cb8de1b0350c)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplaceDatanodeOnFailure.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #714 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/714/ ) HDFS-6101 . TestReplaceDatanodeOnFailure fails occasionally. Contributed (cnauroth: rev 1777608fa075a807c645619fda87cb8de1b0350c) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplaceDatanodeOnFailure.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #725 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/725/)
          HDFS-6101. TestReplaceDatanodeOnFailure fails occasionally. Contributed (cnauroth: rev 1777608fa075a807c645619fda87cb8de1b0350c)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplaceDatanodeOnFailure.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #725 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/725/ ) HDFS-6101 . TestReplaceDatanodeOnFailure fails occasionally. Contributed (cnauroth: rev 1777608fa075a807c645619fda87cb8de1b0350c) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplaceDatanodeOnFailure.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2656 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2656/)
          HDFS-6101. TestReplaceDatanodeOnFailure fails occasionally. Contributed (cnauroth: rev 1777608fa075a807c645619fda87cb8de1b0350c)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplaceDatanodeOnFailure.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2656 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2656/ ) HDFS-6101 . TestReplaceDatanodeOnFailure fails occasionally. Contributed (cnauroth: rev 1777608fa075a807c645619fda87cb8de1b0350c) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplaceDatanodeOnFailure.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #638 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/638/)
          HDFS-6101. TestReplaceDatanodeOnFailure fails occasionally. Contributed (cnauroth: rev 1777608fa075a807c645619fda87cb8de1b0350c)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplaceDatanodeOnFailure.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #638 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/638/ ) HDFS-6101 . TestReplaceDatanodeOnFailure fails occasionally. Contributed (cnauroth: rev 1777608fa075a807c645619fda87cb8de1b0350c) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplaceDatanodeOnFailure.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2576 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2576/)
          HDFS-6101. TestReplaceDatanodeOnFailure fails occasionally. Contributed (cnauroth: rev 1777608fa075a807c645619fda87cb8de1b0350c)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplaceDatanodeOnFailure.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2576 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2576/ ) HDFS-6101 . TestReplaceDatanodeOnFailure fails occasionally. Contributed (cnauroth: rev 1777608fa075a807c645619fda87cb8de1b0350c) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplaceDatanodeOnFailure.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

            People

            • Assignee:
              jojochuang Wei-Chiu Chuang
              Reporter:
              arpitagarwal Arpit Agarwal
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development