Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11745

Increase HDFS test timeouts from 1 second to 10 seconds

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4, 2.8.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      1 second test timeouts are susceptible to failure on overloaded or otherwise slow machines

      1. HDFS-11745.001.patch
        5 kB
        Eric Badger
      2. HDFS-11745.002.patch
        6 kB
        Eric Badger

        Issue Links

          Activity

          Hide
          ebadger Eric Badger added a comment -

          Uploading patch

          Show
          ebadger Eric Badger added a comment - Uploading patch
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 32s Maven dependency ordering for branch
          +1 mvninstall 13m 52s trunk passed
          +1 compile 1m 35s trunk passed
          +1 checkstyle 0m 46s trunk passed
          +1 mvnsite 1m 16s trunk passed
          +1 mvneclipse 0m 30s trunk passed
          -1 findbugs 1m 47s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings.
          +1 javadoc 0m 54s trunk passed
          0 mvndep 0m 7s Maven dependency ordering for patch
          +1 mvninstall 1m 5s the patch passed
          +1 compile 1m 24s the patch passed
          +1 javac 1m 24s the patch passed
          +1 checkstyle 0m 41s the patch passed
          +1 mvnsite 1m 11s the patch passed
          +1 mvneclipse 0m 23s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 19s the patch passed
          +1 javadoc 0m 48s the patch passed
          -1 unit 66m 3s hadoop-hdfs in the patch failed.
          +1 unit 1m 44s hadoop-hdfs-nfs in the patch passed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          99m 34s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
            hadoop.hdfs.server.namenode.TestStartup
            hadoop.hdfs.server.namenode.TestMetadataVersionOutput



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-11745
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12866199/HDFS-11745.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 4856ff90433e 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d4631e4
          Default Java 1.8.0_121
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19295/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19295/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19295/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-nfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19295/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 32s Maven dependency ordering for branch +1 mvninstall 13m 52s trunk passed +1 compile 1m 35s trunk passed +1 checkstyle 0m 46s trunk passed +1 mvnsite 1m 16s trunk passed +1 mvneclipse 0m 30s trunk passed -1 findbugs 1m 47s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings. +1 javadoc 0m 54s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 5s the patch passed +1 compile 1m 24s the patch passed +1 javac 1m 24s the patch passed +1 checkstyle 0m 41s the patch passed +1 mvnsite 1m 11s the patch passed +1 mvneclipse 0m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 19s the patch passed +1 javadoc 0m 48s the patch passed -1 unit 66m 3s hadoop-hdfs in the patch failed. +1 unit 1m 44s hadoop-hdfs-nfs in the patch passed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 99m 34s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration   hadoop.hdfs.server.namenode.TestStartup   hadoop.hdfs.server.namenode.TestMetadataVersionOutput Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11745 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12866199/HDFS-11745.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4856ff90433e 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d4631e4 Default Java 1.8.0_121 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19295/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/19295/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19295/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-nfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19295/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          +1

          Show
          arpitagarwal Arpit Agarwal added a comment - +1
          Hide
          jlowe Jason Lowe added a comment -

          Thanks for the patch! I noticed that TestNameNodeMetrics#testCapacityMetrics also has a pretty low timeout (1.8 seconds, seems like an odd number). I think we should bump that as well.

          Show
          jlowe Jason Lowe added a comment - Thanks for the patch! I noticed that TestNameNodeMetrics#testCapacityMetrics also has a pretty low timeout (1.8 seconds, seems like an odd number). I think we should bump that as well.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Thanks for committing it Jason Lowe. I wrongly assumed Eric had commit privileges.

          Show
          arpitagarwal Arpit Agarwal added a comment - Thanks for committing it Jason Lowe . I wrongly assumed Eric had commit privileges.
          Hide
          jlowe Jason Lowe added a comment -

          Thanks for committing it

          I'm guessing this comment was meant for HADOOP-14377. I haven't committed this one yet – still waiting for Eric's response about the testCapacityMetrics comment.

          Show
          jlowe Jason Lowe added a comment - Thanks for committing it I'm guessing this comment was meant for HADOOP-14377 . I haven't committed this one yet – still waiting for Eric's response about the testCapacityMetrics comment.
          Hide
          ebadger Eric Badger added a comment -

          Thanks for the review, Jason Lowe

          I noticed that TestNameNodeMetrics#testCapacityMetrics also has a pretty low timeout (1.8 seconds, seems like an odd number). I think we should bump that as well.

          Uploading new patch that increases this to 10 seconds as well

          Show
          ebadger Eric Badger added a comment - Thanks for the review, Jason Lowe I noticed that TestNameNodeMetrics#testCapacityMetrics also has a pretty low timeout (1.8 seconds, seems like an odd number). I think we should bump that as well. Uploading new patch that increases this to 10 seconds as well
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          0 mvndep 0m 8s Maven dependency ordering for branch
          +1 mvninstall 14m 42s trunk passed
          +1 compile 1m 53s trunk passed
          +1 checkstyle 0m 47s trunk passed
          +1 mvnsite 1m 26s trunk passed
          +1 mvneclipse 0m 30s trunk passed
          -1 findbugs 1m 53s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings.
          +1 javadoc 0m 51s trunk passed
          0 mvndep 0m 7s Maven dependency ordering for patch
          +1 mvninstall 1m 5s the patch passed
          +1 compile 1m 27s the patch passed
          +1 javac 1m 27s the patch passed
          +1 checkstyle 0m 41s the patch passed
          +1 mvnsite 1m 8s the patch passed
          +1 mvneclipse 0m 25s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 20s the patch passed
          +1 javadoc 0m 48s the patch passed
          -1 unit 72m 45s hadoop-hdfs in the patch failed.
          +1 unit 1m 51s hadoop-hdfs-nfs in the patch passed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          107m 22s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-11745
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867364/HDFS-11745.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 357074d1d20c 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1e71fe8
          Default Java 1.8.0_121
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19385/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19385/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19385/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-nfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19385/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. 0 mvndep 0m 8s Maven dependency ordering for branch +1 mvninstall 14m 42s trunk passed +1 compile 1m 53s trunk passed +1 checkstyle 0m 47s trunk passed +1 mvnsite 1m 26s trunk passed +1 mvneclipse 0m 30s trunk passed -1 findbugs 1m 53s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings. +1 javadoc 0m 51s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 5s the patch passed +1 compile 1m 27s the patch passed +1 javac 1m 27s the patch passed +1 checkstyle 0m 41s the patch passed +1 mvnsite 1m 8s the patch passed +1 mvneclipse 0m 25s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 20s the patch passed +1 javadoc 0m 48s the patch passed -1 unit 72m 45s hadoop-hdfs in the patch failed. +1 unit 1m 51s hadoop-hdfs-nfs in the patch passed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 107m 22s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11745 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867364/HDFS-11745.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 357074d1d20c 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1e71fe8 Default Java 1.8.0_121 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19385/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/19385/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19385/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-nfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19385/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jlowe Jason Lowe added a comment -

          Test failures are unrelated.

          +1 lgtm. Committing this.

          Show
          jlowe Jason Lowe added a comment - Test failures are unrelated. +1 lgtm. Committing this.
          Hide
          jlowe Jason Lowe added a comment -

          Thanks to Eric for the contribution and to Arpit for additional review! I committed this to trunk, branch-2, and branch-2.8.

          Show
          jlowe Jason Lowe added a comment - Thanks to Eric for the contribution and to Arpit for additional review! I committed this to trunk, branch-2, and branch-2.8.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11718 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11718/)
          HDFS-11745. Increase HDFS test timeouts from 1 second to 10 seconds. (jlowe: rev fab73748a1ce982904d332e6ddd46169f186ad54)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUtil.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestRpcProgramNfs3.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestSymlinkHdfsFileSystem.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsTokens.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestSymlinkHdfsFileContext.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11718 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11718/ ) HDFS-11745 . Increase HDFS test timeouts from 1 second to 10 seconds. (jlowe: rev fab73748a1ce982904d332e6ddd46169f186ad54) (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUtil.java (edit) hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestRpcProgramNfs3.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestSymlinkHdfsFileSystem.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsTokens.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestSymlinkHdfsFileContext.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java

            People

            • Assignee:
              ebadger Eric Badger
              Reporter:
              ebadger Eric Badger
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development