Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12542

TestDNS fails on Windows after HADOOP-12437.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: net
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      HADOOP-12437 added several new tests covering functionality of resolving host names based on an alternate network interface. These tests are failing on Windows.

        Issue Links

          Activity

          Hide
          cnauroth Chris Nauroth added a comment -

          I'm attaching a patch. Arpit Agarwal, would you mind reviewing this?

          There are 2 causes for the test failures. First, TestDNS#getLoopbackInterface was using the friendly display name of the interface instead of the interface name. This didn't seem to cause problems on any of my *nix environments, but on Windows, the loopback interface has a friendly name that is not the same as the interface name. These snippets from Scala REPL sessions show the difference. The first is from Mac, and the second is from Windows.

          scala> java.net.NetworkInterface.getByInetAddress(java.net.InetAddress.getLoopbackAddress).getDisplayName
          ckAddress).getDisplayName
          res3: java.lang.String = lo0
          
          scala> java.net.NetworkInterface.getByInetAddress(java.net.InetAddress.getLoopbackAddress).getName
          ckAddress).getName
          res4: java.lang.String = lo0
          
          scala> java.net.NetworkInterface.getByInetAddress(java.net.InetAddress.getLoopbackAddress).getDisplayName
          res2: java.lang.String = Software Loopback Interface 1
          
          scala> java.net.NetworkInterface.getByInetAddress(java.net.InetAddress.getLoopbackAddress).getName
          res3: java.lang.String = lo
          

          The second problem is that even with the above fixed, there was still a failure in TestDNS#testLookupWithHostsFallback. This is because of the infamous problem we've seen many times before on Windows that it will not resolve 127.0.0.1 to host name "localhost", despite the presence of a correct entry in the hosts file. The logic of DNS#getHosts refuses to return an IP address as a host name, so the test fails on Windows. I used assumeTrue to skip this one on Windows.

          With this patch, I verified that TestDNS runs clean on both Mac and Windows.

          Show
          cnauroth Chris Nauroth added a comment - I'm attaching a patch. Arpit Agarwal , would you mind reviewing this? There are 2 causes for the test failures. First, TestDNS#getLoopbackInterface was using the friendly display name of the interface instead of the interface name. This didn't seem to cause problems on any of my *nix environments, but on Windows, the loopback interface has a friendly name that is not the same as the interface name. These snippets from Scala REPL sessions show the difference. The first is from Mac, and the second is from Windows. scala> java.net.NetworkInterface.getByInetAddress(java.net.InetAddress.getLoopbackAddress).getDisplayName ckAddress).getDisplayName res3: java.lang. String = lo0 scala> java.net.NetworkInterface.getByInetAddress(java.net.InetAddress.getLoopbackAddress).getName ckAddress).getName res4: java.lang. String = lo0 scala> java.net.NetworkInterface.getByInetAddress(java.net.InetAddress.getLoopbackAddress).getDisplayName res2: java.lang. String = Software Loopback Interface 1 scala> java.net.NetworkInterface.getByInetAddress(java.net.InetAddress.getLoopbackAddress).getName res3: java.lang. String = lo The second problem is that even with the above fixed, there was still a failure in TestDNS#testLookupWithHostsFallback . This is because of the infamous problem we've seen many times before on Windows that it will not resolve 127.0.0.1 to host name "localhost", despite the presence of a correct entry in the hosts file. The logic of DNS#getHosts refuses to return an IP address as a host name, so the test fails on Windows. I used assumeTrue to skip this one on Windows. With this patch, I verified that TestDNS runs clean on both Mac and Windows.
          Hide
          arpitagarwal Arpit Agarwal added a comment - - edited

          +1 for the patch. I verified the fix on Centos 6, Windows and OS X. Thanks for debugging and fixing it Chris Nauroth!

          It is unfortunate we need to skip that test on Windowsbut I don't see an easy workaround either. I'll try to think of an alternate for Windows but skipping it seems reasonable for now.

          Show
          arpitagarwal Arpit Agarwal added a comment - - edited +1 for the patch. I verified the fix on Centos 6, Windows and OS X. Thanks for debugging and fixing it Chris Nauroth ! It is unfortunate we need to skip that test on Windowsbut I don't see an easy workaround either. I'll try to think of an alternate for Windows but skipping it seems reasonable for now.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 3m 14s trunk passed
          +1 compile 4m 51s trunk passed with JDK v1.8.0_60
          +1 compile 4m 33s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 17s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 45s trunk passed
          +1 javadoc 0m 58s trunk passed with JDK v1.8.0_60
          +1 javadoc 1m 9s trunk passed with JDK v1.7.0_79
          +1 mvninstall 1m 26s the patch passed
          +1 compile 4m 45s the patch passed with JDK v1.8.0_60
          +1 javac 4m 45s the patch passed
          +1 compile 4m 32s the patch passed with JDK v1.7.0_79
          +1 javac 4m 32s the patch passed
          +1 checkstyle 0m 16s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 59s the patch passed
          +1 javadoc 1m 2s the patch passed with JDK v1.8.0_60
          +1 javadoc 1m 8s the patch passed with JDK v1.7.0_79
          -1 unit 7m 27s hadoop-common in the patch failed with JDK v1.8.0_60.
          -1 unit 7m 16s hadoop-common in the patch failed with JDK v1.7.0_79.
          +1 asflicense 0m 25s Patch does not generate ASF License warnings.
          48m 45s



          Reason Tests
          JDK v1.8.0_60 Failed junit tests hadoop.metrics2.impl.TestMetricsSystemImpl
            hadoop.ha.TestZKFailoverController
            hadoop.ipc.TestDecayRpcScheduler
          JDK v1.7.0_79 Failed junit tests hadoop.metrics2.impl.TestMetricsSystemImpl
            hadoop.ipc.TestDecayRpcScheduler



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-03
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770203/HADOOP-12542.001.patch
          JIRA Issue HADOOP-12542
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux 09fd71ba6196 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-1a9afee/precommit/personality/hadoop.sh
          git revision trunk / 5ba2b98
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8004/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8004/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8004/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8004/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8004/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Max memory used 224MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8004/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 3m 14s trunk passed +1 compile 4m 51s trunk passed with JDK v1.8.0_60 +1 compile 4m 33s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 17s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 45s trunk passed +1 javadoc 0m 58s trunk passed with JDK v1.8.0_60 +1 javadoc 1m 9s trunk passed with JDK v1.7.0_79 +1 mvninstall 1m 26s the patch passed +1 compile 4m 45s the patch passed with JDK v1.8.0_60 +1 javac 4m 45s the patch passed +1 compile 4m 32s the patch passed with JDK v1.7.0_79 +1 javac 4m 32s the patch passed +1 checkstyle 0m 16s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 59s the patch passed +1 javadoc 1m 2s the patch passed with JDK v1.8.0_60 +1 javadoc 1m 8s the patch passed with JDK v1.7.0_79 -1 unit 7m 27s hadoop-common in the patch failed with JDK v1.8.0_60. -1 unit 7m 16s hadoop-common in the patch failed with JDK v1.7.0_79. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 48m 45s Reason Tests JDK v1.8.0_60 Failed junit tests hadoop.metrics2.impl.TestMetricsSystemImpl   hadoop.ha.TestZKFailoverController   hadoop.ipc.TestDecayRpcScheduler JDK v1.7.0_79 Failed junit tests hadoop.metrics2.impl.TestMetricsSystemImpl   hadoop.ipc.TestDecayRpcScheduler Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-03 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770203/HADOOP-12542.001.patch JIRA Issue HADOOP-12542 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux 09fd71ba6196 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-1a9afee/precommit/personality/hadoop.sh git revision trunk / 5ba2b98 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8004/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8004/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8004/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8004/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8004/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 224MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8004/console This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          The test failures are unrelated to this test code patch. I haven't been able to repro. I'm going to proceed with the commit later.

          Show
          cnauroth Chris Nauroth added a comment - The test failures are unrelated to this test code patch. I haven't been able to repro. I'm going to proceed with the commit later.
          Hide
          cnauroth Chris Nauroth added a comment -

          I have committed this to trunk and branch-2. Arpit, thank you for the code review.

          Show
          cnauroth Chris Nauroth added a comment - I have committed this to trunk and branch-2. Arpit, thank you for the code review.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #621 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/621/)
          HADOOP-12542. TestDNS fails on Windows after HADOOP-12437. Contributed (cnauroth: rev 957f0311a160afb40dbb0619f455445b4f5d1e32)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestDNS.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #621 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/621/ ) HADOOP-12542 . TestDNS fails on Windows after HADOOP-12437 . Contributed (cnauroth: rev 957f0311a160afb40dbb0619f455445b4f5d1e32) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestDNS.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8747 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8747/)
          HADOOP-12542. TestDNS fails on Windows after HADOOP-12437. Contributed (cnauroth: rev 957f0311a160afb40dbb0619f455445b4f5d1e32)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestDNS.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8747 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8747/ ) HADOOP-12542 . TestDNS fails on Windows after HADOOP-12437 . Contributed (cnauroth: rev 957f0311a160afb40dbb0619f455445b4f5d1e32) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestDNS.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1355 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1355/)
          HADOOP-12542. TestDNS fails on Windows after HADOOP-12437. Contributed (cnauroth: rev 957f0311a160afb40dbb0619f455445b4f5d1e32)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestDNS.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1355 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1355/ ) HADOOP-12542 . TestDNS fails on Windows after HADOOP-12437 . Contributed (cnauroth: rev 957f0311a160afb40dbb0619f455445b4f5d1e32) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestDNS.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #566 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/566/)
          HADOOP-12542. TestDNS fails on Windows after HADOOP-12437. Contributed (cnauroth: rev 957f0311a160afb40dbb0619f455445b4f5d1e32)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestDNS.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #566 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/566/ ) HADOOP-12542 . TestDNS fails on Windows after HADOOP-12437 . Contributed (cnauroth: rev 957f0311a160afb40dbb0619f455445b4f5d1e32) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestDNS.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #632 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/632/)
          HADOOP-12542. TestDNS fails on Windows after HADOOP-12437. Contributed (cnauroth: rev 957f0311a160afb40dbb0619f455445b4f5d1e32)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestDNS.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #632 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/632/ ) HADOOP-12542 . TestDNS fails on Windows after HADOOP-12437 . Contributed (cnauroth: rev 957f0311a160afb40dbb0619f455445b4f5d1e32) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestDNS.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2562 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2562/)
          HADOOP-12542. TestDNS fails on Windows after HADOOP-12437. Contributed (cnauroth: rev 957f0311a160afb40dbb0619f455445b4f5d1e32)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestDNS.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2562 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2562/ ) HADOOP-12542 . TestDNS fails on Windows after HADOOP-12437 . Contributed (cnauroth: rev 957f0311a160afb40dbb0619f455445b4f5d1e32) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestDNS.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2503 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2503/)
          HADOOP-12542. TestDNS fails on Windows after HADOOP-12437. Contributed (cnauroth: rev 957f0311a160afb40dbb0619f455445b4f5d1e32)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestDNS.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2503 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2503/ ) HADOOP-12542 . TestDNS fails on Windows after HADOOP-12437 . Contributed (cnauroth: rev 957f0311a160afb40dbb0619f455445b4f5d1e32) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestDNS.java

            People

            • Assignee:
              cnauroth Chris Nauroth
              Reporter:
              cnauroth Chris Nauroth
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development