Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.22.0
    • 0.22.0
    • test
    • None
    • Reviewed

    Description

      TestBackupNode fails due to unexpected ipv6 address format.

      Attachments

        1. HDFS-1891.part2.patch
          0.5 kB
          Suresh Srinivas
        2. HDFS-1891.patch
          0.6 kB
          Suresh Srinivas

        Issue Links

          Activity

            TestBackupNode failure is due to ipv6 address returned by java Socket.getLocalSocketAddress(). The returned address 0:0:0:0:0:0:0:0:50100.

            We need to disable IPV6 by setting system property java.net.preferIPv4Stack to true,
            as indicated in http://download.oracle.com/javase/6/docs/technotes/guides/net/ipv6_guide/index.html

            2011-05-04 19:52:31,378 INFO  namenode.NameNode (NameNode.java:initialize(410)) - Backup Node up at: 0:0:0:0:0:0:0:0/0:0:0:0:0:0:0:0:50100
            2011-05-04 19:52:31,381 INFO  namenode.Checkpointer (Checkpointer.java:initialize(110)) - Checkpoint Period : 3600 secs (60 min)
            2011-05-04 19:52:31,381 INFO  namenode.Checkpointer (Checkpointer.java:initialize(112)) - Log Size Trigger  : 67108864 bytes (65536 KB)
            2011-05-04 19:52:31,382 INFO  namenode.TestBackupNode (TestBackupNode.java:waitCheckpointDone(95)) - Waiting checkpoint to complete...
            2011-05-04 19:52:31,383 INFO  namenode.FSNamesystem (FSNamesystem.java:startCheckpoint(4548)) - Start checkpoint for 0:0:0:0:0:0:0:0:50100
            2011-05-04 19:52:31,422 INFO  ipc.Server (Server.java:run(1422)) - IPC Server handler 2 on 47860, call startCheckpoint(NamenodeRegistration(0:0:0:0:0:0:0:0:50100, role=Backup Node)), rpc version=1, client version=5, methodsFingerPrint=302283637 from 127.0.0.1:48060: error: java.io.IOException: java.lang.NumberFormatException: For input string: "0:0:0:0:0:0:0:50100"
            java.io.IOException: java.lang.NumberFormatException: For input string: "0:0:0:0:0:0:0:50100"
            
            sureshms Suresh Srinivas added a comment - TestBackupNode failure is due to ipv6 address returned by java Socket.getLocalSocketAddress(). The returned address 0:0:0:0:0:0:0:0:50100. We need to disable IPV6 by setting system property java.net.preferIPv4Stack to true, as indicated in http://download.oracle.com/javase/6/docs/technotes/guides/net/ipv6_guide/index.html 2011-05-04 19:52:31,378 INFO namenode.NameNode (NameNode.java:initialize(410)) - Backup Node up at: 0:0:0:0:0:0:0:0/0:0:0:0:0:0:0:0:50100 2011-05-04 19:52:31,381 INFO namenode.Checkpointer (Checkpointer.java:initialize(110)) - Checkpoint Period : 3600 secs (60 min) 2011-05-04 19:52:31,381 INFO namenode.Checkpointer (Checkpointer.java:initialize(112)) - Log Size Trigger : 67108864 bytes (65536 KB) 2011-05-04 19:52:31,382 INFO namenode.TestBackupNode (TestBackupNode.java:waitCheckpointDone(95)) - Waiting checkpoint to complete... 2011-05-04 19:52:31,383 INFO namenode.FSNamesystem (FSNamesystem.java:startCheckpoint(4548)) - Start checkpoint for 0:0:0:0:0:0:0:0:50100 2011-05-04 19:52:31,422 INFO ipc.Server (Server.java:run(1422)) - IPC Server handler 2 on 47860, call startCheckpoint(NamenodeRegistration(0:0:0:0:0:0:0:0:50100, role=Backup Node)), rpc version=1, client version=5, methodsFingerPrint=302283637 from 127.0.0.1:48060: error: java.io.IOException: java.lang.NumberFormatException: For input string: "0:0:0:0:0:0:0:50100" java.io.IOException: java.lang.NumberFormatException: For input string: "0:0:0:0:0:0:0:50100"

            For junit tests enabling the preferIPv4Stack to true.

            sureshms Suresh Srinivas added a comment - For junit tests enabling the preferIPv4Stack to true.
            mattf Matthew Foley added a comment -

            Is this related to HDFS-1883 or a new issue?

            mattf Matthew Foley added a comment - Is this related to HDFS-1883 or a new issue?

            Searched for TestBackupNode fails and could not find your jira. This is addressing TestBackupNode failure. But the failure is not related to HDFS-1052 change. I will close the other bug as duplicate.

            sureshms Suresh Srinivas added a comment - Searched for TestBackupNode fails and could not find your jira. This is addressing TestBackupNode failure. But the failure is not related to HDFS-1052 change. I will close the other bug as duplicate.

            I started a hudson validation and TestBackupNode succeeds. IPV6 address is no longer returned -

            2011-05-04 21:42:21,530 INFO  namenode.NameNode (NameNode.java:initialize(410)) - Backup Node up at: 0.0.0.0/0.0.0.0:7772
            2011-05-04 21:42:21,542 INFO  namenode.Checkpointer (Checkpointer.java:initialize(110)) - Checkpoint Period : 3600 secs (60 min)
            2011-05-04 21:42:21,542 INFO  namenode.Checkpointer (Checkpointer.java:initialize(112)) - Log Size Trigger  : 67108864 bytes (65536 KB)
            2011-05-04 21:42:21,543 INFO  namenode.NameNode (NameNode.java:errorReport(675)) - Error report from NamenodeRegistration(0.0.0.0:7772, role=Backup Node): Shutting down.
            
            sureshms Suresh Srinivas added a comment - I started a hudson validation and TestBackupNode succeeds. IPV6 address is no longer returned - 2011-05-04 21:42:21,530 INFO namenode.NameNode (NameNode.java:initialize(410)) - Backup Node up at: 0.0.0.0/0.0.0.0:7772 2011-05-04 21:42:21,542 INFO namenode.Checkpointer (Checkpointer.java:initialize(110)) - Checkpoint Period : 3600 secs (60 min) 2011-05-04 21:42:21,542 INFO namenode.Checkpointer (Checkpointer.java:initialize(112)) - Log Size Trigger : 67108864 bytes (65536 KB) 2011-05-04 21:42:21,543 INFO namenode.NameNode (NameNode.java:errorReport(675)) - Error report from NamenodeRegistration(0.0.0.0:7772, role=Backup Node): Shutting down.
            hadoopqa Hadoop QA added a comment -

            -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12478213/HDFS-1891.patch
            against trunk revision 1099525.

            +1 @author. The patch does not contain any @author tags.

            -1 tests included. The patch doesn't appear to include any new or modified tests.
            Please justify why no new tests are needed for this patch.
            Also please list what manual steps were performed to verify this patch.

            +1 javadoc. The javadoc tool did not generate any warning messages.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

            -1 core tests. The patch failed these core unit tests:
            org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
            org.apache.hadoop.hdfs.TestFileConcurrentReader

            +1 contrib tests. The patch passed contrib unit tests.

            +1 system test framework. The patch passed system test framework compile.

            Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/454//testReport/
            Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/454//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
            Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/454//console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12478213/HDFS-1891.patch against trunk revision 1099525. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.TestFileConcurrentReader +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/454//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/454//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/454//console This message is automatically generated.

            I had submitted the patch to Hudson to ensure indeed we are getting IPV4 address. Based on the result posted here, TestBackupNode no longer fails and I also confirmed in the test log that IPV4 address is returned.

            sureshms Suresh Srinivas added a comment - I had submitted the patch to Hudson to ensure indeed we are getting IPV4 address. Based on the result posted here, TestBackupNode no longer fails and I also confirmed in the test log that IPV4 address is returned.
            mattf Matthew Foley added a comment -

            It's worth noting that HADOOP-6056 set java.net.preferIPv4Stack=true in hadoop-config.sh, with the intent of making all Hadoop execution environments use this setting, as of end/Jan/2011, and that need was in part identified by HDFS-1561, which also relates to BackupNode!

            +1 for this proposed patch, which should do the same for HDFS unit tests, which indeed typically do not use hadoop-config.sh.

            Please also open Jiras and submit the corresponding fixes for build.xml in COMMON and MAPRED components, even though failures due to IPv6 have not been detected there yet.

            mattf Matthew Foley added a comment - It's worth noting that HADOOP-6056 set java.net.preferIPv4Stack=true in hadoop-config.sh, with the intent of making all Hadoop execution environments use this setting, as of end/Jan/2011, and that need was in part identified by HDFS-1561 , which also relates to BackupNode! +1 for this proposed patch, which should do the same for HDFS unit tests, which indeed typically do not use hadoop-config.sh. Please also open Jiras and submit the corresponding fixes for build.xml in COMMON and MAPRED components, even though failures due to IPv6 have not been detected there yet.

            Second part of the patch to turn off IPV6 for junit tests in contrib modules.

            sureshms Suresh Srinivas added a comment - Second part of the patch to turn off IPV6 for junit tests in contrib modules.
            mattf Matthew Foley added a comment -

            +1 - and contrib/hdfsproxy/build.xml imports contrib/build-contrib.xml

            mattf Matthew Foley added a comment - +1 - and contrib/hdfsproxy/build.xml imports contrib/build-contrib.xml
            hadoopqa Hadoop QA added a comment -

            -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12478343/HDFS-1891.part2.patch
            against trunk revision 1099972.

            +1 @author. The patch does not contain any @author tags.

            -1 tests included. The patch doesn't appear to include any new or modified tests.
            Please justify why no new tests are needed for this patch.
            Also please list what manual steps were performed to verify this patch.

            +1 javadoc. The javadoc tool did not generate any warning messages.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

            -1 core tests. The patch failed these core unit tests:
            org.apache.hadoop.cli.TestHDFSCLI
            org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
            org.apache.hadoop.hdfs.TestFileConcurrentReader

            +1 contrib tests. The patch passed contrib unit tests.

            +1 system test framework. The patch passed system test framework compile.

            Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/459//testReport/
            Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/459//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
            Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/459//console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12478343/HDFS-1891.part2.patch against trunk revision 1099972. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.TestFileConcurrentReader +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/459//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/459//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/459//console This message is automatically generated.
            hadoopqa Hadoop QA added a comment -

            -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12478344/HDFS-1891.part2.patch
            against trunk revision 1099972.

            +1 @author. The patch does not contain any @author tags.

            -1 tests included. The patch doesn't appear to include any new or modified tests.
            Please justify why no new tests are needed for this patch.
            Also please list what manual steps were performed to verify this patch.

            +1 javadoc. The javadoc tool did not generate any warning messages.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

            -1 core tests. The patch failed these core unit tests:
            org.apache.hadoop.cli.TestHDFSCLI
            org.apache.hadoop.hdfs.TestDatanodeReport
            org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
            org.apache.hadoop.hdfs.TestFileConcurrentReader

            +1 contrib tests. The patch passed contrib unit tests.

            +1 system test framework. The patch passed system test framework compile.

            Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/460//testReport/
            Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/460//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
            Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/460//console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12478344/HDFS-1891.part2.patch against trunk revision 1099972. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.TestDatanodeReport org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.TestFileConcurrentReader +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/460//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/460//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/460//console This message is automatically generated.

            I committed both the patches.

            sureshms Suresh Srinivas added a comment - I committed both the patches.

            0.22 should have the same problem. Could you please commit it to .22

            shv Konstantin Shvachko added a comment - 0.22 should have the same problem. Could you please commit it to .22

            I committed this change to 0.22.

            sureshms Suresh Srinivas added a comment - I committed this change to 0.22.
            hudson Hudson added a comment -

            Integrated in Hadoop-Hdfs-22-branch #49 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/49/)

            hudson Hudson added a comment - Integrated in Hadoop-Hdfs-22-branch #49 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/49/ )
            hudson Hudson added a comment -

            Integrated in Hadoop-Hdfs-trunk #673 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/673/)

            hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #673 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/673/ )

            People

              sureshms Suresh Srinivas
              sureshms Suresh Srinivas
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: