Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13605

Clean up FileSystem javadocs, logging; improve diagnostics on FS load

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: fs
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      We can't easily debug FS instantiation problems as there isn't much detail in what was going on.

      We can add more logging, but cannot simply switch FileSystem.LOG to SLF4J —the class is used in too many places, including tests which cast it. Instead, add a new private SLF4J Logger, LOGGER and switch logging to it.

      While working in the base FileSystem class, take the opportunity to clean up javadocs and comments

      1. add the list of exceptions, including indicating which base classes throw UnsupportedOperationExceptions
      2. cut bits in the comments which are not true

      The outcome of this patch is that IDEs shouldn't highlight most of the file as flawed in some way or another

      1. HADOOP-13605-004.patch
        131 kB
        Steve Loughran
      2. HADOOP-13605-005.patch
        135 kB
        Steve Loughran
      3. HADOOP-13605-branch-2-001.patch
        79 kB
        Steve Loughran
      4. HADOOP-13605-branch-2-002.patch
        132 kB
        Steve Loughran
      5. HADOOP-13605-branch-2-003.patch
        132 kB
        Steve Loughran

        Issue Links

          Activity

          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 001

          code changes are all around FS load diagnostics and logging.
          otherwise: javadoc fixes, some formatting, moving to Java7 <> syntax in constructors.

          Essentially: housekeeping

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 001 code changes are all around FS load diagnostics and logging. otherwise: javadoc fixes, some formatting, moving to Java7 <> syntax in constructors. Essentially: housekeeping
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 16s branch-2 passed
          +1 compile 6m 3s branch-2 passed with JDK v1.8.0_101
          +1 compile 6m 31s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 0m 26s branch-2 passed
          +1 mvnsite 0m 58s branch-2 passed
          +1 mvneclipse 0m 16s branch-2 passed
          +1 findbugs 1m 38s branch-2 passed
          +1 javadoc 0m 45s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 0m 56s branch-2 passed with JDK v1.7.0_111
          +1 mvninstall 0m 42s the patch passed
          +1 compile 5m 24s the patch passed with JDK v1.8.0_101
          +1 javac 5m 24s the patch passed
          +1 compile 6m 31s the patch passed with JDK v1.7.0_111
          +1 javac 6m 31s the patch passed
          -0 checkstyle 0m 27s hadoop-common-project/hadoop-common: The patch generated 8 new + 79 unchanged - 50 fixed = 87 total (was 129)
          +1 mvnsite 0m 57s the patch passed
          +1 mvneclipse 0m 16s the patch passed
          -1 whitespace 0m 0s The patch has 48 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 findbugs 1m 56s the patch passed
          -1 javadoc 0m 47s hadoop-common-project_hadoop-common-jdk1.8.0_101 with JDK v1.8.0_101 generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6)
          -1 javadoc 0m 58s hadoop-common-project_hadoop-common-jdk1.7.0_111 with JDK v1.7.0_111 generated 1 new + 13 unchanged - 0 fixed = 14 total (was 13)
          -1 unit 8m 10s hadoop-common in the patch failed with JDK v1.7.0_111.
          +1 asflicense 0m 23s The patch does not generate ASF License warnings.
          62m 27s



          Reason Tests
          JDK v1.8.0_101 Failed junit tests hadoop.fs.TestFileSystemCaching
          JDK v1.7.0_111 Failed junit tests hadoop.fs.TestFileSystemCaching



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13605
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828308/HADOOP-13605-branch-2-001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 856da69de801 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 55600b3
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10498/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10498/artifact/patchprocess/whitespace-eol.txt
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/10498/artifact/patchprocess/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdk1.8.0_101.txt
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/10498/artifact/patchprocess/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdk1.7.0_111.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10498/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_111.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10498/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10498/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 16s branch-2 passed +1 compile 6m 3s branch-2 passed with JDK v1.8.0_101 +1 compile 6m 31s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 0m 26s branch-2 passed +1 mvnsite 0m 58s branch-2 passed +1 mvneclipse 0m 16s branch-2 passed +1 findbugs 1m 38s branch-2 passed +1 javadoc 0m 45s branch-2 passed with JDK v1.8.0_101 +1 javadoc 0m 56s branch-2 passed with JDK v1.7.0_111 +1 mvninstall 0m 42s the patch passed +1 compile 5m 24s the patch passed with JDK v1.8.0_101 +1 javac 5m 24s the patch passed +1 compile 6m 31s the patch passed with JDK v1.7.0_111 +1 javac 6m 31s the patch passed -0 checkstyle 0m 27s hadoop-common-project/hadoop-common: The patch generated 8 new + 79 unchanged - 50 fixed = 87 total (was 129) +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 16s the patch passed -1 whitespace 0m 0s The patch has 48 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 findbugs 1m 56s the patch passed -1 javadoc 0m 47s hadoop-common-project_hadoop-common-jdk1.8.0_101 with JDK v1.8.0_101 generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6) -1 javadoc 0m 58s hadoop-common-project_hadoop-common-jdk1.7.0_111 with JDK v1.7.0_111 generated 1 new + 13 unchanged - 0 fixed = 14 total (was 13) -1 unit 8m 10s hadoop-common in the patch failed with JDK v1.7.0_111. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 62m 27s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.fs.TestFileSystemCaching JDK v1.7.0_111 Failed junit tests hadoop.fs.TestFileSystemCaching Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13605 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828308/HADOOP-13605-branch-2-001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 856da69de801 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 55600b3 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10498/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10498/artifact/patchprocess/whitespace-eol.txt javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/10498/artifact/patchprocess/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdk1.8.0_101.txt javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/10498/artifact/patchprocess/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdk1.7.0_111.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10498/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_111.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10498/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10498/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Test failed as the error message on a load failure now quotes the scheme, and the test was looking for the exact string.

          org.junit.ComparisonFailure: expected:<...ileSystem for scheme[: null]> but was:<...ileSystem for scheme[ "null"]>
          	at org.junit.Assert.assertEquals(Assert.java:115)
          	at org.junit.Assert.assertEquals(Assert.java:144)
          	at org.apache.hadoop.fs.TestFileSystemCaching.testDefaultFsUris(TestFileSystemCaching.java:99)
          

          as the FS code now throws a specific subclass of IOE, UnsupportedFileSystemException, the message check can be replaced with a catch of that explicit exception type instead.

          Show
          stevel@apache.org Steve Loughran added a comment - Test failed as the error message on a load failure now quotes the scheme, and the test was looking for the exact string. org.junit.ComparisonFailure: expected:<...ileSystem for scheme[: null ]> but was:<...ileSystem for scheme[ " null " ]> at org.junit.Assert.assertEquals(Assert.java:115) at org.junit.Assert.assertEquals(Assert.java:144) at org.apache.hadoop.fs.TestFileSystemCaching.testDefaultFsUris(TestFileSystemCaching.java:99) as the FS code now throws a specific subclass of IOE, UnsupportedFileSystemException , the message check can be replaced with a catch of that explicit exception type instead.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          HADOOP-13605 patch 002:

          • read the javadocs and make them consistent with what happens
          • use consistently FileSystem to refer to instance of the class, filesystem to refer to the place where files live
          • explain that and other things in the introduction
          • trim all trailing whitespace
          • fix up the <p/> references and a few other javadoc issues
          • fix the failing test

          My IDE is mostly happy now, the javadocs are in sync with the FS spec

          Show
          stevel@apache.org Steve Loughran added a comment - HADOOP-13605 patch 002: read the javadocs and make them consistent with what happens use consistently FileSystem to refer to instance of the class, filesystem to refer to the place where files live explain that and other things in the introduction trim all trailing whitespace fix up the <p/> references and a few other javadoc issues fix the failing test My IDE is mostly happy now, the javadocs are in sync with the FS spec
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 28s branch-2 passed
          +1 compile 5m 29s branch-2 passed with JDK v1.8.0_101
          +1 compile 6m 29s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 0m 26s branch-2 passed
          +1 mvnsite 0m 57s branch-2 passed
          +1 mvneclipse 0m 16s branch-2 passed
          +1 findbugs 1m 39s branch-2 passed
          +1 javadoc 0m 45s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 0m 56s branch-2 passed with JDK v1.7.0_111
          +1 mvninstall 0m 41s the patch passed
          +1 compile 5m 24s the patch passed with JDK v1.8.0_101
          +1 javac 5m 24s the patch passed
          +1 compile 6m 47s the patch passed with JDK v1.7.0_111
          +1 javac 6m 47s the patch passed
          -0 checkstyle 0m 26s hadoop-common-project/hadoop-common: The patch generated 7 new + 67 unchanged - 77 fixed = 74 total (was 144)
          +1 mvnsite 1m 2s the patch passed
          +1 mvneclipse 0m 16s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 findbugs 1m 59s the patch passed
          +1 javadoc 0m 49s the patch passed with JDK v1.8.0_101
          +1 javadoc 0m 59s the patch passed with JDK v1.7.0_111
          +1 unit 8m 58s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          62m 13s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13605
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828464/HADOOP-13605-branch-2-002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 061761f947c2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 3f36ac9
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10511/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10511/artifact/patchprocess/whitespace-eol.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10511/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10511/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 28s branch-2 passed +1 compile 5m 29s branch-2 passed with JDK v1.8.0_101 +1 compile 6m 29s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 0m 26s branch-2 passed +1 mvnsite 0m 57s branch-2 passed +1 mvneclipse 0m 16s branch-2 passed +1 findbugs 1m 39s branch-2 passed +1 javadoc 0m 45s branch-2 passed with JDK v1.8.0_101 +1 javadoc 0m 56s branch-2 passed with JDK v1.7.0_111 +1 mvninstall 0m 41s the patch passed +1 compile 5m 24s the patch passed with JDK v1.8.0_101 +1 javac 5m 24s the patch passed +1 compile 6m 47s the patch passed with JDK v1.7.0_111 +1 javac 6m 47s the patch passed -0 checkstyle 0m 26s hadoop-common-project/hadoop-common: The patch generated 7 new + 67 unchanged - 77 fixed = 74 total (was 144) +1 mvnsite 1m 2s the patch passed +1 mvneclipse 0m 16s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 findbugs 1m 59s the patch passed +1 javadoc 0m 49s the patch passed with JDK v1.8.0_101 +1 javadoc 0m 59s the patch passed with JDK v1.7.0_111 +1 unit 8m 58s hadoop-common in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 62m 13s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13605 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828464/HADOOP-13605-branch-2-002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 061761f947c2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 3f36ac9 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10511/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10511/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10511/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10511/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 7s HADOOP-13605 does not apply to branch-2. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue HADOOP-13605
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828464/HADOOP-13605-branch-2-002.patch
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10872/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 7s HADOOP-13605 does not apply to branch-2. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HADOOP-13605 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828464/HADOOP-13605-branch-2-002.patch Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10872/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch 003; in sync with current codebase

          Show
          stevel@apache.org Steve Loughran added a comment - patch 003; in sync with current codebase
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 28s branch-2 passed
          +1 compile 6m 23s branch-2 passed with JDK v1.8.0_101
          +1 compile 6m 45s branch-2 passed with JDK v1.7.0_111
          +1 checkstyle 0m 27s branch-2 passed
          +1 mvnsite 0m 57s branch-2 passed
          +1 mvneclipse 0m 16s branch-2 passed
          +1 findbugs 1m 38s branch-2 passed
          +1 javadoc 0m 44s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 0m 55s branch-2 passed with JDK v1.7.0_111
          +1 mvninstall 0m 39s the patch passed
          +1 compile 6m 11s the patch passed with JDK v1.8.0_101
          +1 javac 6m 11s the patch passed
          +1 compile 6m 47s the patch passed with JDK v1.7.0_111
          -1 javac 6m 47s root-jdk1.7.0_111 with JDK v1.7.0_111 generated 1 new + 949 unchanged - 1 fixed = 950 total (was 950)
          -0 checkstyle 0m 27s hadoop-common-project/hadoop-common: The patch generated 7 new + 67 unchanged - 77 fixed = 74 total (was 144)
          +1 mvnsite 1m 5s the patch passed
          +1 mvneclipse 0m 16s the patch passed
          -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 findbugs 1m 57s the patch passed
          +1 javadoc 0m 50s the patch passed with JDK v1.8.0_101
          +1 javadoc 0m 57s the patch passed with JDK v1.7.0_111
          +1 unit 7m 45s hadoop-common in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          71m 51s



          Reason Tests
          JDK v1.8.0_101 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HADOOP-13605
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834990/HADOOP-13605-branch-2-003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 00e749ca5ee1 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 086577c
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10877/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_111.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10877/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10877/artifact/patchprocess/whitespace-eol.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10877/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10877/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 28s branch-2 passed +1 compile 6m 23s branch-2 passed with JDK v1.8.0_101 +1 compile 6m 45s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 0m 27s branch-2 passed +1 mvnsite 0m 57s branch-2 passed +1 mvneclipse 0m 16s branch-2 passed +1 findbugs 1m 38s branch-2 passed +1 javadoc 0m 44s branch-2 passed with JDK v1.8.0_101 +1 javadoc 0m 55s branch-2 passed with JDK v1.7.0_111 +1 mvninstall 0m 39s the patch passed +1 compile 6m 11s the patch passed with JDK v1.8.0_101 +1 javac 6m 11s the patch passed +1 compile 6m 47s the patch passed with JDK v1.7.0_111 -1 javac 6m 47s root-jdk1.7.0_111 with JDK v1.7.0_111 generated 1 new + 949 unchanged - 1 fixed = 950 total (was 950) -0 checkstyle 0m 27s hadoop-common-project/hadoop-common: The patch generated 7 new + 67 unchanged - 77 fixed = 74 total (was 144) +1 mvnsite 1m 5s the patch passed +1 mvneclipse 0m 16s the patch passed -1 whitespace 0m 0s The patch has 47 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 findbugs 1m 57s the patch passed +1 javadoc 0m 50s the patch passed with JDK v1.8.0_101 +1 javadoc 0m 57s the patch passed with JDK v1.7.0_111 +1 unit 7m 45s hadoop-common in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 71m 51s Reason Tests JDK v1.8.0_101 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HADOOP-13605 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834990/HADOOP-13605-branch-2-003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 00e749ca5ee1 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 086577c Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-HADOOP-Build/10877/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_111.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10877/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/10877/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10877/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10877/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch 004; this is patch 003 reworked for trunk

          Show
          stevel@apache.org Steve Loughran added a comment - patch 004; this is patch 003 reworked for trunk
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 55s trunk passed
          +1 compile 10m 20s trunk passed
          +1 checkstyle 0m 29s trunk passed
          +1 mvnsite 1m 6s trunk passed
          +1 mvneclipse 0m 18s trunk passed
          +1 findbugs 1m 23s trunk passed
          +1 javadoc 0m 47s trunk passed
          +1 mvninstall 0m 37s the patch passed
          +1 compile 9m 46s the patch passed
          +1 javac 9m 46s the patch passed
          -0 checkstyle 0m 31s hadoop-common-project/hadoop-common: The patch generated 8 new + 67 unchanged - 75 fixed = 75 total (was 142)
          +1 mvnsite 1m 11s the patch passed
          +1 mvneclipse 0m 19s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 47s the patch passed
          +1 javadoc 0m 49s the patch passed
          -1 unit 8m 2s hadoop-common in the patch failed.
          +1 asflicense 0m 33s The patch does not generate ASF License warnings.
          47m 4s



          Reason Tests
          Failed junit tests hadoop.fs.TestDefaultUri



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13605
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840129/HADOOP-13605-004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux e69b643dab31 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 3a09e59
          Default Java 1.8.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11116/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11116/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11116/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11116/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 55s trunk passed +1 compile 10m 20s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 1m 6s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 1m 23s trunk passed +1 javadoc 0m 47s trunk passed +1 mvninstall 0m 37s the patch passed +1 compile 9m 46s the patch passed +1 javac 9m 46s the patch passed -0 checkstyle 0m 31s hadoop-common-project/hadoop-common: The patch generated 8 new + 67 unchanged - 75 fixed = 75 total (was 142) +1 mvnsite 1m 11s the patch passed +1 mvneclipse 0m 19s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 49s the patch passed -1 unit 8m 2s hadoop-common in the patch failed. +1 asflicense 0m 33s The patch does not generate ASF License warnings. 47m 4s Reason Tests Failed junit tests hadoop.fs.TestDefaultUri Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13605 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840129/HADOOP-13605-004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e69b643dab31 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3a09e59 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11116/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11116/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11116/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11116/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment - - edited

          Patch 005

          Address checkstyle warnings in source;
          fix test failure by moving to the new lambda test intercept() call, looking for the specific exception and text now raised

          Note there's a couple of places in the production code which could move to lambda expressions too. I've kept them out so that the branch-2 could would match the trunk one; easier to backport an still handle stack-trace-based diags.

          Show
          stevel@apache.org Steve Loughran added a comment - - edited Patch 005 Address checkstyle warnings in source; fix test failure by moving to the new lambda test intercept() call, looking for the specific exception and text now raised Note there's a couple of places in the production code which could move to lambda expressions too. I've kept them out so that the branch-2 could would match the trunk one; easier to backport an still handle stack-trace-based diags.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 6m 46s trunk passed
          +1 compile 9m 34s trunk passed
          +1 checkstyle 0m 30s trunk passed
          +1 mvnsite 1m 1s trunk passed
          +1 mvneclipse 0m 19s trunk passed
          +1 findbugs 1m 25s trunk passed
          +1 javadoc 0m 47s trunk passed
          +1 mvninstall 0m 38s the patch passed
          +1 compile 9m 12s the patch passed
          +1 javac 9m 12s the patch passed
          -0 checkstyle 0m 30s hadoop-common-project/hadoop-common: The patch generated 3 new + 67 unchanged - 75 fixed = 70 total (was 142)
          +1 mvnsite 1m 0s the patch passed
          +1 mvneclipse 0m 19s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 34s the patch passed
          +1 javadoc 0m 48s the patch passed
          +1 unit 8m 25s hadoop-common in the patch passed.
          +1 asflicense 0m 31s The patch does not generate ASF License warnings.
          45m 30s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13605
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840238/HADOOP-13605-005.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 787148505c7a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 3541ed8
          Default Java 1.8.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11122/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11122/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11122/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 6m 46s trunk passed +1 compile 9m 34s trunk passed +1 checkstyle 0m 30s trunk passed +1 mvnsite 1m 1s trunk passed +1 mvneclipse 0m 19s trunk passed +1 findbugs 1m 25s trunk passed +1 javadoc 0m 47s trunk passed +1 mvninstall 0m 38s the patch passed +1 compile 9m 12s the patch passed +1 javac 9m 12s the patch passed -0 checkstyle 0m 30s hadoop-common-project/hadoop-common: The patch generated 3 new + 67 unchanged - 75 fixed = 70 total (was 142) +1 mvnsite 1m 0s the patch passed +1 mvneclipse 0m 19s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 34s the patch passed +1 javadoc 0m 48s the patch passed +1 unit 8m 25s hadoop-common in the patch passed. +1 asflicense 0m 31s The patch does not generate ASF License warnings. 45m 30s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13605 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840238/HADOOP-13605-005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 787148505c7a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3541ed8 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/11122/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11122/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11122/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          I like this patch, especially newly added javadocs. +1

          Show
          liuml07 Mingliang Liu added a comment - I like this patch, especially newly added javadocs. +1
          Hide
          liuml07 Mingliang Liu added a comment -

          Committed to branch-2 and trunk branches. Thanks Steve Loughran for your contribution.

          Show
          liuml07 Mingliang Liu added a comment - Committed to branch-2 and trunk branches. Thanks Steve Loughran for your contribution.
          Hide
          brahmareddy Brahma Reddy Battula added a comment - - edited

          Following test fails after this check-in , Raised HADOOP-13833 to track.
          Pre-commit did not catch because it's from HDFS project TestSymlinkHdfsFileSystem.

          org.junit.ComparisonFailure: expected:<...ileSystem for scheme[: null]> but was:<...ileSystem for scheme[ "null"]>
          	at org.junit.Assert.assertEquals(Assert.java:115)
          	at org.junit.Assert.assertEquals(Assert.java:144)
          	at org.apache.hadoop.fs.SymlinkBaseTest.testCreateLinkUsingPartQualPath2(SymlinkBaseTest.java:574)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:498)
          	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
          	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
          	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
          	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
          	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
          

          REF: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/235/testReport/junit/org.apache.hadoop.fs/TestSymlinkHdfsFileSystem/testCreateLinkUsingPartQualPath2/

          Show
          brahmareddy Brahma Reddy Battula added a comment - - edited Following test fails after this check-in , Raised HADOOP-13833 to track. Pre-commit did not catch because it's from HDFS project TestSymlinkHdfsFileSystem . org.junit.ComparisonFailure: expected:<...ileSystem for scheme[: null]> but was:<...ileSystem for scheme[ "null"]> at org.junit.Assert.assertEquals(Assert.java:115) at org.junit.Assert.assertEquals(Assert.java:144) at org.apache.hadoop.fs.SymlinkBaseTest.testCreateLinkUsingPartQualPath2(SymlinkBaseTest.java:574) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) REF: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/235/testReport/junit/org.apache.hadoop.fs/TestSymlinkHdfsFileSystem/testCreateLinkUsingPartQualPath2/

            People

            • Assignee:
              stevel@apache.org Steve Loughran
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development