Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11353

Improve the unit tests relevant to DataNode volume failure testing

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha2
    • Fix Version/s: 3.0.0-alpha4
    • Component/s: None
    • Labels:
      None

      Description

      Currently there are many tests which start with TestDataNodeVolumeFailure* frequently run timedout or failed. I found one failure test in recent Jenkins building. The stack info:

      org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting.testSuccessiveVolumeFailures
      java.util.concurrent.TimeoutException: Timed out waiting for DN to die
      	at org.apache.hadoop.hdfs.DFSTestUtil.waitForDatanodeDeath(DFSTestUtil.java:702)
      	at org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting.testSuccessiveVolumeFailures(TestDataNodeVolumeFailureReporting.java:208)
      

      The related codes:

          /*
           * Now fail the 2nd volume on the 3rd datanode. All its volumes
           * are now failed and so it should report two volume failures
           * and that it's no longer up. Only wait for two replicas since
           * we'll never get a third.
           */
          DataNodeTestUtils.injectDataDirFailure(dn3Vol2);
          Path file3 = new Path("/test3");
          DFSTestUtil.createFile(fs, file3, 1024, (short)3, 1L);
          DFSTestUtil.waitReplication(fs, file3, (short)2);
      
          // The DN should consider itself dead
          DFSTestUtil.waitForDatanodeDeath(dns.get(2));
      

      Here the code waits for the datanode failed all the volume and then become dead. But it timed out. We would be better to compare that if all the volumes are failed then wair for the datanode dead.

      In addition, we can use the method checkDiskErrorSync to do the disk error check instead of creaing files. In this JIRA, I would like to extract this logic and defined that in DataNodeTestUtils. And then we can reuse this method for datanode volme failure testing in the future.

      1. HDFS-11353.001.patch
        13 kB
        Yiqun Lin
      2. HDFS-11353.002.patch
        10 kB
        Yiqun Lin
      3. HDFS-11353.003.patch
        10 kB
        Yiqun Lin
      4. HDFS-11353.004.patch
        11 kB
        Yiqun Lin
      5. HDFS-11353.005.patch
        13 kB
        Yiqun Lin
      6. HDFS-11353.006.patch
        14 kB
        Yiqun Lin

        Issue Links

          Activity

          Hide
          linyiqun Yiqun Lin added a comment -

          Attach the initial patch. Kindly review.

          Show
          linyiqun Yiqun Lin added a comment - Attach the initial patch. Kindly review.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          +1 mvninstall 13m 15s trunk passed
          +1 compile 0m 44s trunk passed
          +1 checkstyle 0m 28s trunk passed
          +1 mvnsite 0m 51s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 47s trunk passed
          +1 javadoc 0m 41s trunk passed
          +1 mvninstall 0m 46s the patch passed
          +1 compile 0m 42s the patch passed
          +1 javac 0m 42s the patch passed
          -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 82 unchanged - 0 fixed = 83 total (was 82)
          +1 mvnsite 0m 49s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 47s the patch passed
          +1 javadoc 0m 38s the patch passed
          -1 unit 93m 40s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          118m 49s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11353
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848532/HDFS-11353.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 704047c6bd99 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e015b56
          Default Java 1.8.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18227/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18227/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18227/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18227/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 13m 15s trunk passed +1 compile 0m 44s trunk passed +1 checkstyle 0m 28s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 47s trunk passed +1 javadoc 0m 41s trunk passed +1 mvninstall 0m 46s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 82 unchanged - 0 fixed = 83 total (was 82) +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 38s the patch passed -1 unit 93m 40s hadoop-hdfs in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 118m 49s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11353 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848532/HDFS-11353.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 704047c6bd99 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e015b56 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18227/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18227/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18227/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18227/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          manojg Manoj Govindassamy added a comment - - edited

          Thanks for working on this Yiqun Lin.

          • In testSuccessiveVolumeFailures, by placing DataNodeTestUtils.checkDiskErrorSync() we are making it to synchronously wait for the error check to complete which was otherwise not there before.
          •     DataNodeTestUtils.injectDataDirFailure(dn1Vol1, dn2Vol1);
                Path file1 = new Path("/test1");
                DFSTestUtil.createFile(fs, file1, 1024, (short)3, 1L);
                DFSTestUtil.waitReplication(fs, file1, (short)3);
            

            All these injectDataDirFailure are followed by createFile. In my test create file on failed volume is immediately caught by handleDiskError which is what would DataNodeTestUtils.checkDiskErrorSync() also do, right ? So, createFile in a way simulating the needed checkDiskErrorSync() ?

          Show
          manojg Manoj Govindassamy added a comment - - edited Thanks for working on this Yiqun Lin . In testSuccessiveVolumeFailures , by placing DataNodeTestUtils.checkDiskErrorSync() we are making it to synchronously wait for the error check to complete which was otherwise not there before. DataNodeTestUtils.injectDataDirFailure(dn1Vol1, dn2Vol1); Path file1 = new Path("/test1"); DFSTestUtil.createFile(fs, file1, 1024, (short)3, 1L); DFSTestUtil.waitReplication(fs, file1, (short)3); All these injectDataDirFailure are followed by createFile. In my test create file on failed volume is immediately caught by handleDiskError which is what would DataNodeTestUtils.checkDiskErrorSync() also do, right ? So, createFile in a way simulating the needed checkDiskErrorSync() ?
          Hide
          linyiqun Yiqun Lin added a comment - - edited

          Thanks Manoj Govindassamy for the comments. Yes, you are right. Actually, creating file will also trigger the handleDiskError logic. So I found one antoher way to improve the logic for the test TestDataNodeVolumeFailureReporting.testSuccessiveVolumeFailures.

          In the original logic, it fails the 2nd volume on the 3rd datanode then immediately wait the datanode to be dead. If every thing goes well, it should satisify the condition. But in failure case, it runs error. So one way I am thinking that can be improved is that we should do the failed volumes check. Only when all the volume being failed, then the datanode will be shutdown.

          In addition, in the latest patch I keep the change for checkDiskErrorSync since that can be reused in the future and be good for us to test. I have removed necessary change as well.

          Show
          linyiqun Yiqun Lin added a comment - - edited Thanks Manoj Govindassamy for the comments. Yes, you are right. Actually, creating file will also trigger the handleDiskError logic. So I found one antoher way to improve the logic for the test TestDataNodeVolumeFailureReporting.testSuccessiveVolumeFailures . In the original logic, it fails the 2nd volume on the 3rd datanode then immediately wait the datanode to be dead. If every thing goes well, it should satisify the condition. But in failure case, it runs error. So one way I am thinking that can be improved is that we should do the failed volumes check. Only when all the volume being failed, then the datanode will be shutdown. In addition, in the latest patch I keep the change for checkDiskErrorSync since that can be reused in the future and be good for us to test. I have removed necessary change as well.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 7m 25s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          +1 mvninstall 19m 41s trunk passed
          +1 compile 1m 8s trunk passed
          +1 checkstyle 0m 39s trunk passed
          +1 mvnsite 1m 12s trunk passed
          +1 mvneclipse 0m 17s trunk passed
          +1 findbugs 2m 17s trunk passed
          +1 javadoc 0m 49s trunk passed
          +1 mvninstall 1m 4s the patch passed
          +1 compile 0m 58s the patch passed
          +1 javac 0m 58s the patch passed
          +1 checkstyle 0m 32s the patch passed
          +1 mvnsite 1m 3s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 17s the patch passed
          +1 javadoc 0m 44s the patch passed
          -1 unit 98m 2s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          140m 18s



          Reason Tests
          Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2
            org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11353
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848711/HDFS-11353.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 5e7846886053 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 98c35bb
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18232/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18232/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18232/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 7m 25s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 19m 41s trunk passed +1 compile 1m 8s trunk passed +1 checkstyle 0m 39s trunk passed +1 mvnsite 1m 12s trunk passed +1 mvneclipse 0m 17s trunk passed +1 findbugs 2m 17s trunk passed +1 javadoc 0m 49s trunk passed +1 mvninstall 1m 4s the patch passed +1 compile 0m 58s the patch passed +1 javac 0m 58s the patch passed +1 checkstyle 0m 32s the patch passed +1 mvnsite 1m 3s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 17s the patch passed +1 javadoc 0m 44s the patch passed -1 unit 98m 2s hadoop-hdfs in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 140m 18s Reason Tests Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2   org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11353 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848711/HDFS-11353.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5e7846886053 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 98c35bb Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18232/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18232/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18232/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          linyiqun Yiqun Lin added a comment -

          Reupload the v002 patch with adding the timeout time. It will be help for us to find the timed out test after Jenkins building.

          Show
          linyiqun Yiqun Lin added a comment - Reupload the v002 patch with adding the timeout time. It will be help for us to find the timed out test after Jenkins building.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          +1 mvninstall 14m 37s trunk passed
          +1 compile 0m 53s trunk passed
          +1 checkstyle 0m 28s trunk passed
          +1 mvnsite 0m 51s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 43s trunk passed
          +1 javadoc 0m 39s trunk passed
          +1 mvninstall 0m 45s the patch passed
          +1 compile 0m 42s the patch passed
          +1 javac 0m 42s the patch passed
          +1 checkstyle 0m 24s the patch passed
          +1 mvnsite 0m 47s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 46s the patch passed
          +1 javadoc 0m 37s the patch passed
          -1 unit 63m 52s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          90m 15s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11353
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848905/HDFS-11353.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c47202fcef37 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 3fa0d54
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18241/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18241/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18241/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 14m 37s trunk passed +1 compile 0m 53s trunk passed +1 checkstyle 0m 28s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 43s trunk passed +1 javadoc 0m 39s trunk passed +1 mvninstall 0m 45s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed +1 checkstyle 0m 24s the patch passed +1 mvnsite 0m 47s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 37s the patch passed -1 unit 63m 52s hadoop-hdfs in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 90m 15s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11353 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848905/HDFS-11353.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c47202fcef37 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3fa0d54 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18241/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18241/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18241/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          manojg Manoj Govindassamy added a comment -

          Looks like TestDataNodeVolumeFailureReporting is still failing with timeout. This time its stuck on triggering heratbeat on the 3rd DN.

          Show
          manojg Manoj Govindassamy added a comment - Looks like TestDataNodeVolumeFailureReporting is still failing with timeout. This time its stuck on triggering heratbeat on the 3rd DN.
          Hide
          linyiqun Yiqun Lin added a comment -

          Yes, it looks stranger here. I'm not so sure about the reason of this. Attach a patch without triggering heartbeat. I think we can file another JIRA to have a track if it still fails. At least, we can know why the test TestDataNodeVolumeFailureReporting fails so frequently.

          Show
          linyiqun Yiqun Lin added a comment - Yes, it looks stranger here. I'm not so sure about the reason of this. Attach a patch without triggering heartbeat. I think we can file another JIRA to have a track if it still fails. At least, we can know why the test TestDataNodeVolumeFailureReporting fails so frequently.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          +1 mvninstall 14m 43s trunk passed
          +1 compile 0m 49s trunk passed
          +1 checkstyle 0m 29s trunk passed
          +1 mvnsite 1m 3s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 58s trunk passed
          +1 javadoc 0m 43s trunk passed
          +1 mvninstall 0m 54s the patch passed
          +1 compile 0m 53s the patch passed
          +1 javac 0m 53s the patch passed
          +1 checkstyle 0m 27s the patch passed
          +1 mvnsite 0m 59s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 7s the patch passed
          +1 javadoc 0m 39s the patch passed
          -1 unit 115m 18s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 37s The patch does not generate ASF License warnings.
          143m 46s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestReadStripedFileWithMissingBlocks
            hadoop.hdfs.web.TestWebHDFS
          Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11353
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12849035/HDFS-11353.003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux b33b04153ed2 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a2c5012
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18246/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18246/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18246/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 14m 43s trunk passed +1 compile 0m 49s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 1m 3s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 58s trunk passed +1 javadoc 0m 43s trunk passed +1 mvninstall 0m 54s the patch passed +1 compile 0m 53s the patch passed +1 javac 0m 53s the patch passed +1 checkstyle 0m 27s the patch passed +1 mvnsite 0m 59s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 7s the patch passed +1 javadoc 0m 39s the patch passed -1 unit 115m 18s hadoop-hdfs in the patch failed. +1 asflicense 0m 37s The patch does not generate ASF License warnings. 143m 46s Reason Tests Failed junit tests hadoop.hdfs.TestReadStripedFileWithMissingBlocks   hadoop.hdfs.web.TestWebHDFS Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11353 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12849035/HDFS-11353.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b33b04153ed2 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a2c5012 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18246/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18246/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18246/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          linyiqun Yiqun Lin added a comment -

          Attach the v004 patch with adding timeout for TestDataNodeVolumeFailure as well.

          Show
          linyiqun Yiqun Lin added a comment - Attach the v004 patch with adding timeout for TestDataNodeVolumeFailure as well.
          Hide
          xiaochen Xiao Chen added a comment -

          Thanks Yiqun Lin for the work, good to see test improvements!

          Didn't look into the TestDataNodeVolumeFailureReporting case yet, but some general comments/questions.

          • since DataNodeTestUtils#checkDiskErrorSync is really loop-waiting, maybe better to rename it like waitForDistError.
          • same method, could use GenericTestUtils.waitFor to replace the while loop + assertTrue.
          • In the test classes, instead of adding specific timeout to each test case, we can just add a @Rule for the timeout, to the entire test class. This is also a little more future proof. The only downside is we need to examine all existing test cases, to make sure this timeout isn't too aggressive.
          Show
          xiaochen Xiao Chen added a comment - Thanks Yiqun Lin for the work, good to see test improvements! Didn't look into the TestDataNodeVolumeFailureReporting case yet, but some general comments/questions. since DataNodeTestUtils#checkDiskErrorSync is really loop-waiting, maybe better to rename it like waitForDistError . same method, could use GenericTestUtils.waitFor to replace the while loop + assertTrue . In the test classes, instead of adding specific timeout to each test case, we can just add a @Rule for the timeout, to the entire test class. This is also a little more future proof. The only downside is we need to examine all existing test cases, to make sure this timeout isn't too aggressive.
          Hide
          linyiqun Yiqun Lin added a comment -

          Thanks Xiao Chen for taking a look for this and giving your comments. The comments seem great.
          Attach a new patch to address the comments. I add the timeout @Rule in class TestDataNodeVolumeFailureToleration as well since I found TestDataNodeVolumeFailureToleration failed sometimes also. I set the timeout as 120s as you mentioned in HDFS-11372 and this will be a sufficient time. I took a look in the recent Jenkins buildings, the relevant tests just cost around 1~2minutes.

          TestDataNodeVolumeFailure	1 分 7 秒	0	-1	0		10	+1	10	
          TestDataNodeVolumeFailureReporting	1 分 35 秒	0		0		6	+6	6	+6
          TestDataNodeVolumeFailureToleration	43 秒	0		0		4		4
          

          If the test still fails, we will be easily caught and can file the new JIRA to have a track.
          Thanks for the review.

          Show
          linyiqun Yiqun Lin added a comment - Thanks Xiao Chen for taking a look for this and giving your comments. The comments seem great. Attach a new patch to address the comments. I add the timeout @Rule in class TestDataNodeVolumeFailureToleration as well since I found TestDataNodeVolumeFailureToleration failed sometimes also. I set the timeout as 120s as you mentioned in HDFS-11372 and this will be a sufficient time. I took a look in the recent Jenkins buildings, the relevant tests just cost around 1~2minutes. TestDataNodeVolumeFailure 1 分 7 秒 0 -1 0 10 +1 10 TestDataNodeVolumeFailureReporting 1 分 35 秒 0 0 6 +6 6 +6 TestDataNodeVolumeFailureToleration 43 秒 0 0 4 4 If the test still fails, we will be easily caught and can file the new JIRA to have a track. Thanks for the review.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          +1 mvninstall 14m 5s trunk passed
          +1 compile 0m 50s trunk passed
          +1 checkstyle 0m 29s trunk passed
          +1 mvnsite 0m 54s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 48s trunk passed
          +1 javadoc 0m 41s trunk passed
          +1 mvninstall 0m 48s the patch passed
          +1 compile 0m 52s the patch passed
          +1 javac 0m 52s the patch passed
          -0 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 82 unchanged - 0 fixed = 85 total (was 82)
          +1 mvnsite 0m 55s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 48s the patch passed
          +1 javadoc 0m 36s the patch passed
          -1 unit 69m 14s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 33s The patch does not generate ASF License warnings.
          95m 57s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11353
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12849797/HDFS-11353.005.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux b664257ce34c 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 312b36d
          Default Java 1.8.0_121
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18287/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18287/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18287/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18287/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 14m 5s trunk passed +1 compile 0m 50s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 48s trunk passed +1 javadoc 0m 41s trunk passed +1 mvninstall 0m 48s the patch passed +1 compile 0m 52s the patch passed +1 javac 0m 52s the patch passed -0 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 82 unchanged - 0 fixed = 85 total (was 82) +1 mvnsite 0m 55s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 48s the patch passed +1 javadoc 0m 36s the patch passed -1 unit 69m 14s hadoop-hdfs in the patch failed. +1 asflicense 0m 33s The patch does not generate ASF License warnings. 95m 57s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11353 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12849797/HDFS-11353.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b664257ce34c 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 312b36d Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18287/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18287/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18287/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18287/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          linyiqun Yiqun Lin added a comment -

          The failed test is not related. Will attach a new patch to fix checkstyle warnings. But before attaching the new patch, I'd like to let Xiao Chen for taking a quick look for the latest patch. Hi Xiao, does the patch look good to you?

          Show
          linyiqun Yiqun Lin added a comment - The failed test is not related. Will attach a new patch to fix checkstyle warnings. But before attaching the new patch, I'd like to let Xiao Chen for taking a quick look for the latest patch. Hi Xiao, does the patch look good to you?
          Hide
          xiaochen Xiao Chen added a comment -

          Thanks Yiqun Lin, +1 pending:

          1. The check in DataNodeTestUtils can be simplified to just return dn.getLastDiskErrorCheck() != lastDiskErrorCheck

                  if (dn.getLastDiskErrorCheck() == lastDiskErrorCheck) {
                    return false;
                  }
          
                  return true;
          

          2. Suggest we add a log when entering DataNodeTestUtils#waitForDiskError to make debugging easier.

          3. I'm not sure about the move in TestDataNodeVolumeFailureReporting#testSuccessiveVolumeFailures, and ideally we should see if we can repro the failure here. But if that's very hard to come by, I think moving that down is okay.

          Show
          xiaochen Xiao Chen added a comment - Thanks Yiqun Lin , +1 pending: 1. The check in DataNodeTestUtils can be simplified to just return dn.getLastDiskErrorCheck() != lastDiskErrorCheck if (dn.getLastDiskErrorCheck() == lastDiskErrorCheck) { return false ; } return true ; 2. Suggest we add a log when entering DataNodeTestUtils#waitForDiskError to make debugging easier. 3. I'm not sure about the move in TestDataNodeVolumeFailureReporting#testSuccessiveVolumeFailures , and ideally we should see if we can repro the failure here. But if that's very hard to come by, I think moving that down is okay.
          Hide
          linyiqun Yiqun Lin added a comment -

          Thanks Xiao Chen for the review!

          and ideally we should see if we can repro the failure here

          It's hard to reproduce the failure in my local. So I catch the failure info from the Jenkins buildings. Actually, I'm also not so sure why the timeout happened in DFSTestUtil.waitForDatanodeDeath when running TestDataNodeVolumeFailureReporting#testSuccessiveVolumeFailures. But I know there is one chance that will cause this when not all the disks are failure. So I make a minor change to ensure all the disks are failure before wait datanode to death. I will make the further track if the test still fails.

          Attach a new patch to address the other two comments.

          Show
          linyiqun Yiqun Lin added a comment - Thanks Xiao Chen for the review! and ideally we should see if we can repro the failure here It's hard to reproduce the failure in my local. So I catch the failure info from the Jenkins buildings. Actually, I'm also not so sure why the timeout happened in DFSTestUtil.waitForDatanodeDeath when running TestDataNodeVolumeFailureReporting#testSuccessiveVolumeFailures . But I know there is one chance that will cause this when not all the disks are failure. So I make a minor change to ensure all the disks are failure before wait datanode to death. I will make the further track if the test still fails. Attach a new patch to address the other two comments.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          +1 mvninstall 15m 31s trunk passed
          +1 compile 0m 52s trunk passed
          +1 checkstyle 0m 30s trunk passed
          +1 mvnsite 0m 57s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 2m 0s trunk passed
          +1 javadoc 0m 45s trunk passed
          +1 mvninstall 0m 59s the patch passed
          +1 compile 0m 53s the patch passed
          +1 javac 0m 53s the patch passed
          +1 checkstyle 0m 28s the patch passed
          +1 mvnsite 0m 59s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 25s the patch passed
          +1 javadoc 0m 41s the patch passed
          -1 unit 76m 40s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          106m 20s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11353
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12850531/HDFS-11353.006.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 32f9194a9ce6 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6aa09dc
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18314/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18314/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18314/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 15m 31s trunk passed +1 compile 0m 52s trunk passed +1 checkstyle 0m 30s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 2m 0s trunk passed +1 javadoc 0m 45s trunk passed +1 mvninstall 0m 59s the patch passed +1 compile 0m 53s the patch passed +1 javac 0m 53s the patch passed +1 checkstyle 0m 28s the patch passed +1 mvnsite 0m 59s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 25s the patch passed +1 javadoc 0m 41s the patch passed -1 unit 76m 40s hadoop-hdfs in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 106m 20s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11353 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12850531/HDFS-11353.006.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 32f9194a9ce6 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6aa09dc Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18314/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18314/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18314/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment -

          Thanks Yiqun Lin, tracking that in another jira SGTM. +1 on patch 6.

          Show
          xiaochen Xiao Chen added a comment - Thanks Yiqun Lin , tracking that in another jira SGTM. +1 on patch 6.
          Hide
          linyiqun Yiqun Lin added a comment - - edited

          The failure test is not related. Committed to trunk. Thanks Manoj Govindassamy and Xiao Chen for the reviews on this!

          Show
          linyiqun Yiqun Lin added a comment - - edited The failure test is not related. Committed to trunk. Thanks Manoj Govindassamy and Xiao Chen for the reviews on this!
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11206 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11206/)
          HDFS-11353. Improve the unit tests relevant to DataNode volume failure (yqlin: rev 3433f572fafb9a7d7608915f3471fa6f025fa75c)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeHotSwapVolumes.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailureReporting.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailureToleration.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11206 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11206/ ) HDFS-11353 . Improve the unit tests relevant to DataNode volume failure (yqlin: rev 3433f572fafb9a7d7608915f3471fa6f025fa75c) (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeHotSwapVolumes.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailureReporting.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailureToleration.java
          Hide
          jzhuge John Zhuge added a comment -

          Not sure it is related, got the timeouts in TestDataNodeVolumeFailure and TestDataNodeVolumeFailureReporting:

            TestDataNodeVolumeFailure.testUnderReplicationAfterVolFailure:412 » Timeout Ti...
            TestDataNodeVolumeFailureReporting.tearDown:102->Object.wait:-2 »  test timed ...
          

          https://builds.apache.org/job/PreCommit-HDFS-Build/18334/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt

          Show
          jzhuge John Zhuge added a comment - Not sure it is related, got the timeouts in TestDataNodeVolumeFailure and TestDataNodeVolumeFailureReporting : TestDataNodeVolumeFailure.testUnderReplicationAfterVolFailure:412 » Timeout Ti... TestDataNodeVolumeFailureReporting.tearDown:102->Object.wait:-2 » test timed ... https://builds.apache.org/job/PreCommit-HDFS-Build/18334/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Hide
          linyiqun Yiqun Lin added a comment -

          Hi, John Zhuge, will tracking that in another JIRA. Thanks for catching this!

          Show
          linyiqun Yiqun Lin added a comment - Hi, John Zhuge , will tracking that in another JIRA. Thanks for catching this!
          Hide
          linyiqun Yiqun Lin added a comment - - edited

          Had filed the JIRA for one test failure TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailur in HDFS-11398. I made the concrete analysis there and made a fix.

          Show
          linyiqun Yiqun Lin added a comment - - edited Had filed the JIRA for one test failure TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailur in HDFS-11398 . I made the concrete analysis there and made a fix.

            People

            • Assignee:
              linyiqun Yiqun Lin
              Reporter:
              linyiqun Yiqun Lin
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development