Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      TestBackupNode fails due to unexpected ipv6 address format.

      1. HDFS-1891.part2.patch
        0.5 kB
        Suresh Srinivas
      2. HDFS-1891.patch
        0.6 kB
        Suresh Srinivas

        Issue Links

          Activity

          Hide
          Suresh Srinivas added a comment -

          TestBackupNode failure is due to ipv6 address returned by java Socket.getLocalSocketAddress(). The returned address 0:0:0:0:0:0:0:0:50100.

          We need to disable IPV6 by setting system property java.net.preferIPv4Stack to true,
          as indicated in http://download.oracle.com/javase/6/docs/technotes/guides/net/ipv6_guide/index.html

          2011-05-04 19:52:31,378 INFO  namenode.NameNode (NameNode.java:initialize(410)) - Backup Node up at: 0:0:0:0:0:0:0:0/0:0:0:0:0:0:0:0:50100
          2011-05-04 19:52:31,381 INFO  namenode.Checkpointer (Checkpointer.java:initialize(110)) - Checkpoint Period : 3600 secs (60 min)
          2011-05-04 19:52:31,381 INFO  namenode.Checkpointer (Checkpointer.java:initialize(112)) - Log Size Trigger  : 67108864 bytes (65536 KB)
          2011-05-04 19:52:31,382 INFO  namenode.TestBackupNode (TestBackupNode.java:waitCheckpointDone(95)) - Waiting checkpoint to complete...
          2011-05-04 19:52:31,383 INFO  namenode.FSNamesystem (FSNamesystem.java:startCheckpoint(4548)) - Start checkpoint for 0:0:0:0:0:0:0:0:50100
          2011-05-04 19:52:31,422 INFO  ipc.Server (Server.java:run(1422)) - IPC Server handler 2 on 47860, call startCheckpoint(NamenodeRegistration(0:0:0:0:0:0:0:0:50100, role=Backup Node)), rpc version=1, client version=5, methodsFingerPrint=302283637 from 127.0.0.1:48060: error: java.io.IOException: java.lang.NumberFormatException: For input string: "0:0:0:0:0:0:0:50100"
          java.io.IOException: java.lang.NumberFormatException: For input string: "0:0:0:0:0:0:0:50100"
          
          Show
          Suresh Srinivas added a comment - TestBackupNode failure is due to ipv6 address returned by java Socket.getLocalSocketAddress(). The returned address 0:0:0:0:0:0:0:0:50100. We need to disable IPV6 by setting system property java.net.preferIPv4Stack to true, as indicated in http://download.oracle.com/javase/6/docs/technotes/guides/net/ipv6_guide/index.html 2011-05-04 19:52:31,378 INFO namenode.NameNode (NameNode.java:initialize(410)) - Backup Node up at: 0:0:0:0:0:0:0:0/0:0:0:0:0:0:0:0:50100 2011-05-04 19:52:31,381 INFO namenode.Checkpointer (Checkpointer.java:initialize(110)) - Checkpoint Period : 3600 secs (60 min) 2011-05-04 19:52:31,381 INFO namenode.Checkpointer (Checkpointer.java:initialize(112)) - Log Size Trigger : 67108864 bytes (65536 KB) 2011-05-04 19:52:31,382 INFO namenode.TestBackupNode (TestBackupNode.java:waitCheckpointDone(95)) - Waiting checkpoint to complete... 2011-05-04 19:52:31,383 INFO namenode.FSNamesystem (FSNamesystem.java:startCheckpoint(4548)) - Start checkpoint for 0:0:0:0:0:0:0:0:50100 2011-05-04 19:52:31,422 INFO ipc.Server (Server.java:run(1422)) - IPC Server handler 2 on 47860, call startCheckpoint(NamenodeRegistration(0:0:0:0:0:0:0:0:50100, role=Backup Node)), rpc version=1, client version=5, methodsFingerPrint=302283637 from 127.0.0.1:48060: error: java.io.IOException: java.lang.NumberFormatException: For input string: "0:0:0:0:0:0:0:50100" java.io.IOException: java.lang.NumberFormatException: For input string: "0:0:0:0:0:0:0:50100"
          Hide
          Suresh Srinivas added a comment -

          For junit tests enabling the preferIPv4Stack to true.

          Show
          Suresh Srinivas added a comment - For junit tests enabling the preferIPv4Stack to true.
          Hide
          Matt Foley added a comment -

          Is this related to HDFS-1883 or a new issue?

          Show
          Matt Foley added a comment - Is this related to HDFS-1883 or a new issue?
          Hide
          Suresh Srinivas added a comment -

          Searched for TestBackupNode fails and could not find your jira. This is addressing TestBackupNode failure. But the failure is not related to HDFS-1052 change. I will close the other bug as duplicate.

          Show
          Suresh Srinivas added a comment - Searched for TestBackupNode fails and could not find your jira. This is addressing TestBackupNode failure. But the failure is not related to HDFS-1052 change. I will close the other bug as duplicate.
          Hide
          Suresh Srinivas added a comment -

          I started a hudson validation and TestBackupNode succeeds. IPV6 address is no longer returned -

          2011-05-04 21:42:21,530 INFO  namenode.NameNode (NameNode.java:initialize(410)) - Backup Node up at: 0.0.0.0/0.0.0.0:7772
          2011-05-04 21:42:21,542 INFO  namenode.Checkpointer (Checkpointer.java:initialize(110)) - Checkpoint Period : 3600 secs (60 min)
          2011-05-04 21:42:21,542 INFO  namenode.Checkpointer (Checkpointer.java:initialize(112)) - Log Size Trigger  : 67108864 bytes (65536 KB)
          2011-05-04 21:42:21,543 INFO  namenode.NameNode (NameNode.java:errorReport(675)) - Error report from NamenodeRegistration(0.0.0.0:7772, role=Backup Node): Shutting down.
          
          Show
          Suresh Srinivas added a comment - I started a hudson validation and TestBackupNode succeeds. IPV6 address is no longer returned - 2011-05-04 21:42:21,530 INFO namenode.NameNode (NameNode.java:initialize(410)) - Backup Node up at: 0.0.0.0/0.0.0.0:7772 2011-05-04 21:42:21,542 INFO namenode.Checkpointer (Checkpointer.java:initialize(110)) - Checkpoint Period : 3600 secs (60 min) 2011-05-04 21:42:21,542 INFO namenode.Checkpointer (Checkpointer.java:initialize(112)) - Log Size Trigger : 67108864 bytes (65536 KB) 2011-05-04 21:42:21,543 INFO namenode.NameNode (NameNode.java:errorReport(675)) - Error report from NamenodeRegistration(0.0.0.0:7772, role=Backup Node): Shutting down.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12478213/HDFS-1891.patch
          against trunk revision 1099525.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
          org.apache.hadoop.hdfs.TestFileConcurrentReader

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/454//testReport/
          Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/454//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/454//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12478213/HDFS-1891.patch against trunk revision 1099525. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.TestFileConcurrentReader +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/454//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/454//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/454//console This message is automatically generated.
          Hide
          Suresh Srinivas added a comment -

          I had submitted the patch to Hudson to ensure indeed we are getting IPV4 address. Based on the result posted here, TestBackupNode no longer fails and I also confirmed in the test log that IPV4 address is returned.

          Show
          Suresh Srinivas added a comment - I had submitted the patch to Hudson to ensure indeed we are getting IPV4 address. Based on the result posted here, TestBackupNode no longer fails and I also confirmed in the test log that IPV4 address is returned.
          Hide
          Matt Foley added a comment -

          It's worth noting that HADOOP-6056 set java.net.preferIPv4Stack=true in hadoop-config.sh, with the intent of making all Hadoop execution environments use this setting, as of end/Jan/2011, and that need was in part identified by HDFS-1561, which also relates to BackupNode!

          +1 for this proposed patch, which should do the same for HDFS unit tests, which indeed typically do not use hadoop-config.sh.

          Please also open Jiras and submit the corresponding fixes for build.xml in COMMON and MAPRED components, even though failures due to IPv6 have not been detected there yet.

          Show
          Matt Foley added a comment - It's worth noting that HADOOP-6056 set java.net.preferIPv4Stack=true in hadoop-config.sh, with the intent of making all Hadoop execution environments use this setting, as of end/Jan/2011, and that need was in part identified by HDFS-1561 , which also relates to BackupNode! +1 for this proposed patch, which should do the same for HDFS unit tests, which indeed typically do not use hadoop-config.sh. Please also open Jiras and submit the corresponding fixes for build.xml in COMMON and MAPRED components, even though failures due to IPv6 have not been detected there yet.
          Hide
          Suresh Srinivas added a comment -

          Second part of the patch to turn off IPV6 for junit tests in contrib modules.

          Show
          Suresh Srinivas added a comment - Second part of the patch to turn off IPV6 for junit tests in contrib modules.
          Hide
          Matt Foley added a comment -

          +1 - and contrib/hdfsproxy/build.xml imports contrib/build-contrib.xml

          Show
          Matt Foley added a comment - +1 - and contrib/hdfsproxy/build.xml imports contrib/build-contrib.xml
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12478343/HDFS-1891.part2.patch
          against trunk revision 1099972.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.cli.TestHDFSCLI
          org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
          org.apache.hadoop.hdfs.TestFileConcurrentReader

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/459//testReport/
          Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/459//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/459//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12478343/HDFS-1891.part2.patch against trunk revision 1099972. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.TestFileConcurrentReader +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/459//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/459//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/459//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12478344/HDFS-1891.part2.patch
          against trunk revision 1099972.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.cli.TestHDFSCLI
          org.apache.hadoop.hdfs.TestDatanodeReport
          org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
          org.apache.hadoop.hdfs.TestFileConcurrentReader

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/460//testReport/
          Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/460//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/460//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12478344/HDFS-1891.part2.patch against trunk revision 1099972. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.TestDatanodeReport org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.TestFileConcurrentReader +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/460//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/460//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/460//console This message is automatically generated.
          Hide
          Suresh Srinivas added a comment -

          I committed both the patches.

          Show
          Suresh Srinivas added a comment - I committed both the patches.
          Hide
          Konstantin Shvachko added a comment -

          0.22 should have the same problem. Could you please commit it to .22

          Show
          Konstantin Shvachko added a comment - 0.22 should have the same problem. Could you please commit it to .22
          Hide
          Suresh Srinivas added a comment -

          I committed this change to 0.22.

          Show
          Suresh Srinivas added a comment - I committed this change to 0.22.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-22-branch #49 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/49/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-22-branch #49 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/49/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #673 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/673/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #673 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/673/ )

            People

            • Assignee:
              Suresh Srinivas
              Reporter:
              Suresh Srinivas
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development