Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-6008

Namenode dead node link is giving HTTP error 500

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.4.0
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Clicking on the link to bring up list of dead nodes throws IllegalArgumentException on the Namenode web UI.
      The screenshot is attached.

      1. nndeadnodelist.png
        503 kB
        Benoy Antony
      2. HDFS-6008.patch
        2 kB
        Benoy Antony
      3. HDFS-6008.patch
        2 kB
        Benoy Antony
      4. HDFS-6008.patch
        2 kB
        Benoy Antony

        Activity

        Hide
        Suresh Srinivas added a comment -

        Is Affects Version/s 2.2.0 or 2.4.0 with the new webUI?

        Show
        Suresh Srinivas added a comment - Is Affects Version/s 2.2.0 or 2.4.0 with the new webUI?
        Hide
        Benoy Antony added a comment -

        A few hostname in the hosts file did resolve properly to their ip address. This resulted in the HostFileManager creating entries with empty ip address.
        When generating the deadnode list, the nodes with empty ip address caused IllegalAugumentException which was displayed on the screen.

        Attaching a patch which doesn't canonicalize if the ip ip address is null. A Unit test with positive and negative cases is also added.

        Show
        Benoy Antony added a comment - A few hostname in the hosts file did resolve properly to their ip address. This resulted in the HostFileManager creating entries with empty ip address. When generating the deadnode list, the nodes with empty ip address caused IllegalAugumentException which was displayed on the screen. Attaching a patch which doesn't canonicalize if the ip ip address is null. A Unit test with positive and negative cases is also added.
        Hide
        Benoy Antony added a comment -

        Suresh Srinivas Not sure whether it affects new webui. But the bug is present in the trunk as well. The attached patch is for trunk.

        Show
        Benoy Antony added a comment - Suresh Srinivas Not sure whether it affects new webui. But the bug is present in the trunk as well. The attached patch is for trunk.
        Hide
        Benoy Antony added a comment -

        Typo corrected in the previous comment:
        A few hostnames in the hosts file did NOT resolve properly to their ip address. This resulted in the HostFileManager creating entries with empty ip address.
        When generating the deadnode list, the nodes with empty ip address caused IllegalArgumentException which was displayed on the screen.
        Attaching a patch which doesn't canonicalize if the ip address is null/empty. A Unit test with positive and negative cases is also added.

        Show
        Benoy Antony added a comment - Typo corrected in the previous comment: A few hostnames in the hosts file did NOT resolve properly to their ip address. This resulted in the HostFileManager creating entries with empty ip address. When generating the deadnode list, the nodes with empty ip address caused IllegalArgumentException which was displayed on the screen. Attaching a patch which doesn't canonicalize if the ip address is null/empty. A Unit test with positive and negative cases is also added.
        Hide
        Suresh Srinivas added a comment -

        Marking this as a blocker for 2.4.0

        Show
        Suresh Srinivas added a comment - Marking this as a blocker for 2.4.0
        Hide
        Haohui Mai added a comment -

        This should not be a blocker since it does not seem to affect the new web UI, which is the default.

        Show
        Haohui Mai added a comment - This should not be a blocker since it does not seem to affect the new web UI, which is the default.
        Hide
        Haohui Mai added a comment -

        Changing the target version to 2.2.0 since it is required for the older releases which the new web ui is unavailable.

        Show
        Haohui Mai added a comment - Changing the target version to 2.2.0 since it is required for the older releases which the new web ui is unavailable.
        Hide
        Benoy Antony added a comment -

        Verified that the new webui (trunk) does NOT throw exception in this case. But legacy UI on trunk throws the exception.

        Show
        Benoy Antony added a comment - Verified that the new webui (trunk) does NOT throw exception in this case. But legacy UI on trunk throws the exception.
        Hide
        Chris Nauroth added a comment -

        Hi, Benoy Antony. Nice find. I wondered if it might be a bug in HostFileManager to allow creation of entries with empty IP addresses, but after re-reading the code, I can see that this is by design.

        The current patch contains tab characters. The project standard is to indent by 2 spaces. Would you please change that? After that, I think this patch is ready.

        Show
        Chris Nauroth added a comment - Hi, Benoy Antony . Nice find. I wondered if it might be a bug in HostFileManager to allow creation of entries with empty IP addresses, but after re-reading the code, I can see that this is by design. The current patch contains tab characters. The project standard is to indent by 2 spaces. Would you please change that? After that, I think this patch is ready.
        Hide
        Benoy Antony added a comment -

        Thanks for the review, Chris Nauroth. Attaching the patch without tabs.

        Show
        Benoy Antony added a comment - Thanks for the review, Chris Nauroth . Attaching the patch without tabs.
        Hide
        Chris Nauroth added a comment -

        +1 for the patch, pending Jenkins. Thanks for cleaning up the tabs.

        Show
        Chris Nauroth added a comment - +1 for the patch, pending Jenkins. Thanks for cleaning up the tabs.
        Hide
        Benoy Antony added a comment -

        Thanks Chris Nauroth.
        Attaching a patch with slight improvement as the first patch results in two assignments in most cases.
        In the new patch , assignment is done only once.

        Show
        Benoy Antony added a comment - Thanks Chris Nauroth . Attaching a patch with slight improvement as the first patch results in two assignments in most cases. In the new patch , assignment is done only once.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12630828/HDFS-6008.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6231//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6231//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12630828/HDFS-6008.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6231//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6231//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12631013/HDFS-6008.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6233//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6233//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12631013/HDFS-6008.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6233//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6233//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12631014/HDFS-6008.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6236//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6236//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12631014/HDFS-6008.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6236//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6236//console This message is automatically generated.
        Hide
        Chris Nauroth added a comment -

        I committed this to trunk, branch-2 and branch-2.4. Benoy, thank you for reporting the bug and contributing a patch.

        The target version had been set to 2.2.0. I'm not aware of any plans for further dot releases on the 2.2 or 2.3 line. (There is no mention in the roadmap wiki.) If a dot release happens later, then this is something that could be merged down easily.

        Show
        Chris Nauroth added a comment - I committed this to trunk, branch-2 and branch-2.4. Benoy, thank you for reporting the bug and contributing a patch. The target version had been set to 2.2.0. I'm not aware of any plans for further dot releases on the 2.2 or 2.3 line. (There is no mention in the roadmap wiki.) If a dot release happens later, then this is something that could be merged down easily.
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #5225 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5225/)
        HDFS-6008. Namenode dead node link is giving HTTP error 500. Contributed by Benoy Antony. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1571881)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestJspHelper.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #5225 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5225/ ) HDFS-6008 . Namenode dead node link is giving HTTP error 500. Contributed by Benoy Antony. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1571881 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestJspHelper.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #493 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/493/)
        HDFS-6008. Namenode dead node link is giving HTTP error 500. Contributed by Benoy Antony. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1571881)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestJspHelper.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #493 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/493/ ) HDFS-6008 . Namenode dead node link is giving HTTP error 500. Contributed by Benoy Antony. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1571881 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestJspHelper.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Hdfs-trunk #1685 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1685/)
        HDFS-6008. Namenode dead node link is giving HTTP error 500. Contributed by Benoy Antony. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1571881)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestJspHelper.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1685 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1685/ ) HDFS-6008 . Namenode dead node link is giving HTTP error 500. Contributed by Benoy Antony. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1571881 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestJspHelper.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1710 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1710/)
        HDFS-6008. Namenode dead node link is giving HTTP error 500. Contributed by Benoy Antony. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1571881)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestJspHelper.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1710 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1710/ ) HDFS-6008 . Namenode dead node link is giving HTTP error 500. Contributed by Benoy Antony. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1571881 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/TestJspHelper.java

          People

          • Assignee:
            Benoy Antony
            Reporter:
            Benoy Antony
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development