Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1623 High Availability Framework for HDFS NN
  3. HDFS-2859

LOCAL_ADDRESS_MATCHER.match has NPE when called from DFSUtil.getSuffixIDs when the host is incorrect

        Details

        • Type: Sub-task
        • Status: Resolved
        • Priority: Minor
        • Resolution: Fixed
        • Affects Version/s: HA branch (HDFS-1623)
        • Fix Version/s: HA branch (HDFS-1623)
        • Component/s: ha, namenode
        • Labels:
          None

          Issue Links

          incorporates

          Sub-task - The sub-task of the issue HDFS-2858 DFSUtil.getSuffixIDs silently ignores exception in NetUtils.createSocketAddr

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            bikassaha Bikas Saha added a comment -

            I was testing startup of an HA name node with an incorrect host and it crashed with an NPE here.

            Show
            bikassaha Bikas Saha added a comment - I was testing startup of an HA name node with an incorrect host and it crashed with an NPE here.
            Hide
            Permalink
            bikassaha Bikas Saha added a comment -

            Attaching a patch with a simple fix for the NPE.

            A general question I have is whether it is ok for certain name node hosts to be unresolved during startup. I guess it might be ok because some machines might legitimately be out of service and disconnected from the network.

            Show
            bikassaha Bikas Saha added a comment - Attaching a patch with a simple fix for the NPE. A general question I have is whether it is ok for certain name node hosts to be unresolved during startup. I guess it might be ok because some machines might legitimately be out of service and disconnected from the network.
            Hide
            Permalink
            bikassaha Bikas Saha added a comment -

            Based on the response to my question above I may resolve HDFS-2858 with this patch. If its ok to have erroneous hosts then it might work to simply log the exception and proceed.

            Show
            bikassaha Bikas Saha added a comment - Based on the response to my question above I may resolve HDFS-2858 with this patch. If its ok to have erroneous hosts then it might work to simply log the exception and proceed.
            Hide
            Permalink
            tlipcon Todd Lipcon added a comment -

            A general question I have is whether it is ok for certain name node hosts to be unresolved during startup. I guess it might be ok because some machines might legitimately be out of service and disconnected from the network.

            I don't think it's OK – though a loud warning might be enough. Even if the host is down/disconnected, we'd expect the domain name to be around.

            The patch here looks good, though. +1

            Show
            tlipcon Todd Lipcon added a comment - A general question I have is whether it is ok for certain name node hosts to be unresolved during startup. I guess it might be ok because some machines might legitimately be out of service and disconnected from the network. I don't think it's OK – though a loud warning might be enough. Even if the host is down/disconnected, we'd expect the domain name to be around. The patch here looks good, though. +1
            Hide
            Permalink
            tlipcon Todd Lipcon added a comment -

            I committed this. I also fixed an extra unneeded import of NNStorage which was in the patch before commit. In the future, can you also please attach "p0" patches rather than p1? ie use git diff --no-prefix? Thanks!

            Show
            tlipcon Todd Lipcon added a comment - I committed this. I also fixed an extra unneeded import of NNStorage which was in the patch before commit. In the future, can you also please attach "p0" patches rather than p1? ie use git diff --no-prefix? Thanks!
            Hide
            Permalink
            bikassaha Bikas Saha added a comment -

            Thanks!

            Show
            bikassaha Bikas Saha added a comment - Thanks!
            Hide
            Permalink
            hudson Hudson added a comment -

            Integrated in Hadoop-Hdfs-HAbranch-build #66 (See https://builds.apache.org/job/Hadoop-Hdfs-HAbranch-build/66/)
            HDFS-2859. LOCAL_ADDRESS_MATCHER.match has NPE when called from DFSUtil.getSuffixIDs when the host is incorrect. Contributed by Bikas Saha.

            todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1239356
            Files :

            • /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.HDFS-1623.txt
            • /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
            Show
            hudson Hudson added a comment - Integrated in Hadoop-Hdfs-HAbranch-build #66 (See https://builds.apache.org/job/Hadoop-Hdfs-HAbranch-build/66/ ) HDFS-2859 . LOCAL_ADDRESS_MATCHER.match has NPE when called from DFSUtil.getSuffixIDs when the host is incorrect. Contributed by Bikas Saha. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1239356 Files : /hadoop/common/branches/ HDFS-1623 /hadoop-hdfs-project/hadoop-hdfs/CHANGES. HDFS-1623 .txt /hadoop/common/branches/ HDFS-1623 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java

              People

              • Assignee:
                bikassaha Bikas Saha
                Reporter:
                bikassaha Bikas Saha
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development