Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2086

If the include hosts list contains host name, after restarting namenode, datanodes registrant is denied

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: namenode
    • Labels:
      None

      Description

      As the title describes the problem: if the include host list contains host name, after restarting namenodes, the datanodes registrant is denied by namenodes. This is because after namenode is restarted, the still alive data node will try to register itself with the namenode and it identifies itself with its IP address. However, namenode only allows all the hosts in its hosts list to registrant and all of them are hostnames. So namenode would deny the datanode registration.

      1. HDFS-2086.patch
        10 kB
        Tanping Wang
      2. HDFS-2086.7.patch
        9 kB
        Tanping Wang
      3. HDFS-2086.6.patch
        9 kB
        Tanping Wang
      4. HDFS-2086.5.patch
        10 kB
        Tanping Wang
      5. HDFS-2086.4.patch
        11 kB
        Tanping Wang
      6. HDFS-2086.3.patch
        10 kB
        Tanping Wang
      7. HDFS-2086.2.patch
        10 kB
        Tanping Wang

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #705 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/705/)
        HDFS-2086. If the include hosts list contains host names, after restarting namenode, data nodes registration is denied. Contributed by Tanping Wang.

        tanping : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139090
        Files :

        • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hdfs/CHANGES.txt
        • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestStartup.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #705 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/705/ ) HDFS-2086 . If the include hosts list contains host names, after restarting namenode, data nodes registration is denied. Contributed by Tanping Wang. tanping : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139090 Files : /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hdfs/CHANGES.txt /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestStartup.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #755 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/755/)
        HDFS-2086. If the include hosts list contains host names, after restarting namenode, data nodes registration is denied. Contributed by Tanping Wang.

        tanping : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139090
        Files :

        • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hdfs/CHANGES.txt
        • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestStartup.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #755 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/755/ ) HDFS-2086 . If the include hosts list contains host names, after restarting namenode, data nodes registration is denied. Contributed by Tanping Wang. tanping : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139090 Files : /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hdfs/CHANGES.txt /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestStartup.java
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12483638/HDFS-2086.7.patch
        against trunk revision 1138645.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/830//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/830//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/830//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12483638/HDFS-2086.7.patch against trunk revision 1138645. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/830//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/830//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/830//console This message is automatically generated.
        Hide
        Tanping Wang added a comment -

        Address the review comments.

        Show
        Tanping Wang added a comment - Address the review comments.
        Hide
        Jitendra Nath Pandey added a comment -

        Patch looks good to me

        nit: In checkInList, can we avoid repeating (iaddr != null)?

        +1 otherwise.

        Show
        Jitendra Nath Pandey added a comment - Patch looks good to me nit: In checkInList, can we avoid repeating (iaddr != null)? +1 otherwise.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12483519/HDFS-2086.5.patch
        against trunk revision 1138645.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/822//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/822//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/822//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12483519/HDFS-2086.5.patch against trunk revision 1138645. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/822//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/822//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/822//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12483384/HDFS-2086.4.patch
        against trunk revision 1138262.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/813//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/813//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/813//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12483384/HDFS-2086.4.patch against trunk revision 1138262. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/813//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/813//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/813//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12483349/HDFS-2086.3.patch
        against trunk revision 1138098.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/811//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/811//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/811//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12483349/HDFS-2086.3.patch against trunk revision 1138098. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/811//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/811//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/811//console This message is automatically generated.
        Hide
        Tanping Wang added a comment -

        Upload a new patch to fix the failed unit test case.

        Show
        Tanping Wang added a comment - Upload a new patch to fix the failed unit test case.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12483317/HDFS-2086.2.patch
        against trunk revision 1138098.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:
        org.apache.hadoop.hdfs.server.namenode.TestBlocksWithNotEnoughRacks

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/807//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/807//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/807//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12483317/HDFS-2086.2.patch against trunk revision 1138098. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.server.namenode.TestBlocksWithNotEnoughRacks +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/807//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/807//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/807//console This message is automatically generated.
        Hide
        Tanping Wang added a comment -

        Upload a new patch to address the review comments.

        Show
        Tanping Wang added a comment - Upload a new patch to address the review comments.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12483221/HDFS-2086.patch
        against trunk revision 1137675.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:
        org.apache.hadoop.hdfs.server.namenode.TestStartup

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/802//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/802//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/802//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12483221/HDFS-2086.patch against trunk revision 1137675. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.server.namenode.TestStartup +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/802//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/802//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/802//console This message is automatically generated.
        Hide
        Jitendra Nath Pandey added a comment -

        1. inHostsList and inExcludeHostsList do same things on two different lists. Both can use a single method that also takes the list as argument.
        2. Do we really need to look into hostsList for both node.getName and iaddr.getHostName? I understand node.getName may actually be returning the ip:port, but for IP iaddr.getHostAddress is more reliable. Caveat with the later approach: Can we assume ipAddr and node (DatanodeID) will always be for the same host?

        Minor: Indentation in checkIncludeListForDead.

        Show
        Jitendra Nath Pandey added a comment - 1. inHostsList and inExcludeHostsList do same things on two different lists. Both can use a single method that also takes the list as argument. 2. Do we really need to look into hostsList for both node.getName and iaddr.getHostName? I understand node.getName may actually be returning the ip:port, but for IP iaddr.getHostAddress is more reliable. Caveat with the later approach: Can we assume ipAddr and node (DatanodeID) will always be for the same host? Minor: Indentation in checkIncludeListForDead.
        Hide
        Tanping Wang added a comment -

        There is a second part of this problem. When namenode checks datanode status by calling
        FSNameSystem#getDatanodeListForReport

        The namenode goes over its include hosts list and tries to determine if all the hosts in the include list are all registrant. If not, the host will be added into the dead list. In our case, as just mentioned early, after the namenode restarted, datanode registrants itself with its IP address. But the include list still
        contains its host name. So the hostname is not recognized by namenode and is added into the dead list when namenode reports the datanodes status.

        Show
        Tanping Wang added a comment - There is a second part of this problem. When namenode checks datanode status by calling FSNameSystem#getDatanodeListForReport The namenode goes over its include hosts list and tries to determine if all the hosts in the include list are all registrant. If not, the host will be added into the dead list. In our case, as just mentioned early, after the namenode restarted, datanode registrants itself with its IP address . But the include list still contains its host name . So the hostname is not recognized by namenode and is added into the dead list when namenode reports the datanodes status.

          People

          • Assignee:
            Tanping Wang
            Reporter:
            Tanping Wang
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development