Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11212

NetUtils.wrapException to handle SocketException explicitly

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.2
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: util
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      the NetUtil.wrapException()} method doesn't catch {{SocketException, so it is wrapped with an IOE —this loses information, and stops any extra diags /wiki links being added

      1. HADOOP-11212-002.patch
        6 kB
        Steve Loughran
      2. HADOOP-11212-001.patch
        1 kB
        Steve Loughran

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9550 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9550/)
          HADOOP-11212. NetUtils.wrapException to handle SocketException (arp: rev 7280550a8f668df8aa32e4630db4ead49e9b8b6d)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestNetUtils.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/NetUtils.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9550 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9550/ ) HADOOP-11212 . NetUtils.wrapException to handle SocketException (arp: rev 7280550a8f668df8aa32e4630db4ead49e9b8b6d) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestNetUtils.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/NetUtils.java
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          I've committed this for 2.8.0.

          Show
          arpitagarwal Arpit Agarwal added a comment - I've committed this for 2.8.0.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          +1 for the v002 patch, and it has a rare Jenkins +1.

          Show
          arpitagarwal Arpit Agarwal added a comment - +1 for the v002 patch, and it has a rare Jenkins +1.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 30s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 42s trunk passed
          +1 compile 5m 58s trunk passed with JDK v1.8.0_74
          +1 compile 6m 35s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 21s trunk passed
          +1 mvnsite 0m 58s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 31s trunk passed
          +1 javadoc 0m 54s trunk passed with JDK v1.8.0_74
          +1 javadoc 1m 4s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 41s the patch passed
          +1 compile 5m 38s the patch passed with JDK v1.8.0_74
          +1 javac 5m 38s the patch passed
          +1 compile 6m 41s the patch passed with JDK v1.7.0_95
          +1 javac 6m 41s the patch passed
          +1 checkstyle 0m 20s hadoop-common-project/hadoop-common: patch generated 0 new + 38 unchanged - 1 fixed = 38 total (was 39)
          +1 mvnsite 0m 54s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 48s the patch passed
          +1 javadoc 0m 54s the patch passed with JDK v1.8.0_74
          +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95
          +1 unit 7m 6s hadoop-common in the patch passed with JDK v1.8.0_74.
          +1 unit 7m 16s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 23s Patch does not generate ASF License warnings.
          58m 54s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791508/HADOOP-11212-002.patch
          JIRA Issue HADOOP-11212
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux a2026e1089b5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 3e8099a
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8790/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8790/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 30s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 42s trunk passed +1 compile 5m 58s trunk passed with JDK v1.8.0_74 +1 compile 6m 35s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 58s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 31s trunk passed +1 javadoc 0m 54s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 4s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 41s the patch passed +1 compile 5m 38s the patch passed with JDK v1.8.0_74 +1 javac 5m 38s the patch passed +1 compile 6m 41s the patch passed with JDK v1.7.0_95 +1 javac 6m 41s the patch passed +1 checkstyle 0m 20s hadoop-common-project/hadoop-common: patch generated 0 new + 38 unchanged - 1 fixed = 38 total (was 39) +1 mvnsite 0m 54s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 48s the patch passed +1 javadoc 0m 54s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95 +1 unit 7m 6s hadoop-common in the patch passed with JDK v1.8.0_74. +1 unit 7m 16s hadoop-common in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 58m 54s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791508/HADOOP-11212-002.patch JIRA Issue HADOOP-11212 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a2026e1089b5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3e8099a Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8790/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8790/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -
          1. adds that test
          2. cleans up some of the indentation of multiline arguments that were way off to the right of the IDE window
          3. hardened the assert checking of one test, to use the assertInException assertion
          Show
          stevel@apache.org Steve Loughran added a comment - adds that test cleans up some of the indentation of multiline arguments that were way off to the right of the IDE window hardened the assert checking of one test, to use the assertInException assertion
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 33s trunk passed
          +1 compile 6m 1s trunk passed with JDK v1.8.0_72
          +1 compile 6m 41s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 21s trunk passed
          +1 mvnsite 1m 2s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 35s trunk passed
          +1 javadoc 0m 51s trunk passed with JDK v1.8.0_72
          +1 javadoc 1m 2s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 41s the patch passed
          +1 compile 5m 57s the patch passed with JDK v1.8.0_72
          +1 javac 5m 57s the patch passed
          +1 compile 6m 42s the patch passed with JDK v1.7.0_95
          +1 javac 6m 42s the patch passed
          +1 checkstyle 0m 21s the patch passed
          +1 mvnsite 1m 0s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 49s the patch passed
          +1 javadoc 0m 54s the patch passed with JDK v1.8.0_72
          +1 javadoc 1m 4s the patch passed with JDK v1.7.0_95
          -1 unit 16m 30s hadoop-common in the patch failed with JDK v1.8.0_72.
          +1 unit 7m 16s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 21s Patch does not generate ASF License warnings.
          68m 20s



          Reason Tests
          JDK v1.8.0_72 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790942/HADOOP-11212-001.patch
          JIRA Issue HADOOP-11212
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 2aa1f4f981a7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 27e0681
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8765/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8765/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8765/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8765/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 33s trunk passed +1 compile 6m 1s trunk passed with JDK v1.8.0_72 +1 compile 6m 41s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 1m 2s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 35s trunk passed +1 javadoc 0m 51s trunk passed with JDK v1.8.0_72 +1 javadoc 1m 2s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 41s the patch passed +1 compile 5m 57s the patch passed with JDK v1.8.0_72 +1 javac 5m 57s the patch passed +1 compile 6m 42s the patch passed with JDK v1.7.0_95 +1 javac 6m 42s the patch passed +1 checkstyle 0m 21s the patch passed +1 mvnsite 1m 0s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 49s the patch passed +1 javadoc 0m 54s the patch passed with JDK v1.8.0_72 +1 javadoc 1m 4s the patch passed with JDK v1.7.0_95 -1 unit 16m 30s hadoop-common in the patch failed with JDK v1.8.0_72. +1 unit 7m 16s hadoop-common in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 68m 20s Reason Tests JDK v1.8.0_72 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790942/HADOOP-11212-001.patch JIRA Issue HADOOP-11212 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2aa1f4f981a7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 27e0681 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8765/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8765/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8765/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8765/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 001, adds the exception. Some of the exceptions above (e.g NoRouteToHostException) are subclasses of SocketException —so that must go last. A comment emphasises that point.

          I would include a new stack trace, but SLIDER-1096 covers how I can't build/test against Hadoop 2.8.0 right now

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 001, adds the exception. Some of the exceptions above (e.g NoRouteToHostException) are subclasses of SocketException —so that must go last. A comment emphasises that point. I would include a new stack trace, but SLIDER-1096 covers how I can't build/test against Hadoop 2.8.0 right now
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Page for this is on the wiki: http://wiki.apache.org/hadoop/SocketException

          Show
          stevel@apache.org Steve Loughran added a comment - Page for this is on the wiki: http://wiki.apache.org/hadoop/SocketException
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Stack when trying to talk to a local-subnet-VM that's down.

          
          

          Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 22.766 sec <<< FAILURE! - in org.apache.slider.funtest.lifecycle.AgentClusterLifecycleIT
          org.apache.slider.funtest.lifecycle.AgentClusterLifecycleIT Time elapsed: 22.765 sec <<< ERROR!
          java.io.IOException: Failed on local exception: java.net.SocketException: Host is down; Host Details : local host is: "stevel-763.local/240.0.0.1"; destination host is: "nn.example.com":8020;
          at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764)
          at org.apache.hadoop.ipc.Client.call(Client.java:1472)
          at org.apache.hadoop.ipc.Client.call(Client.java:1399)
          at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
          at com.sun.proxy.$Proxy22.getFileInfo(Unknown Source)
          at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:752)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:606)
          at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
          at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
          at com.sun.proxy.$Proxy23.getFileInfo(Unknown Source)
          at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1957)
          at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1118)
          at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114)
          at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
          at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114)
          at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
          at org.apache.slider.funtest.framework.FileUploader.mkHomeDir(FileUploader.groovy:123)
          at org.apache.slider.funtest.framework.FileUploader$mkHomeDir.call(Unknown Source)
          at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
          at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
          at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
          at org.apache.slider.funtest.framework.AgentCommandTestBase.setupAgent(AgentCommandTestBase.groovy:105)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          {code]

          Show
          stevel@apache.org Steve Loughran added a comment - Stack when trying to talk to a local-subnet-VM that's down. Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 22.766 sec <<< FAILURE! - in org.apache.slider.funtest.lifecycle.AgentClusterLifecycleIT org.apache.slider.funtest.lifecycle.AgentClusterLifecycleIT Time elapsed: 22.765 sec <<< ERROR! java.io.IOException: Failed on local exception: java.net.SocketException: Host is down; Host Details : local host is: "stevel-763.local/240.0.0.1"; destination host is: "nn.example.com":8020; at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764) at org.apache.hadoop.ipc.Client.call(Client.java:1472) at org.apache.hadoop.ipc.Client.call(Client.java:1399) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at com.sun.proxy.$Proxy22.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:752) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy23.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1957) at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1118) at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114) at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400) at org.apache.slider.funtest.framework.FileUploader.mkHomeDir(FileUploader.groovy:123) at org.apache.slider.funtest.framework.FileUploader$mkHomeDir.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112) at org.apache.slider.funtest.framework.AgentCommandTestBase.setupAgent(AgentCommandTestBase.groovy:105) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) {code]

            People

            • Assignee:
              stevel@apache.org Steve Loughran
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development