Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4344

dfshealth.jsp throws NumberFormatException when dfs.hosts/dfs.hosts.exclude includes port number

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.2-alpha, 3.0.0-alpha1
    • 2.0.3-alpha
    • namenode
    • None
    • Reviewed

    Description

      dfs.hosts and dfs.hosts.exclude files cannot contain a port number of host.
      If contained, and access a dfshealth.jsp on a webui, we got a NumberFormatException.

      How to reproduce:

      $ cat /tmp/include.txt
      salve-host1:9999
      
      $ cat /tmp/exclude.txt
      slave-host1:9999
      
      $ hdfs namenode -Ddfs.hosts=/tmp/include.txt -Ddfs.hosts.exclude=/tmp/exclude.txt
      

      Error:

      Problem accessing /dfshealth.jsp. Reason:
      
          For input string: ":9999"
      Caused by:
      
      java.lang.NumberFormatException: For input string: ":9999"
           at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
           at java.lang.Integer.parseInt(Integer.java:449)
           at java.lang.Integer.valueOf(Integer.java:554)
           at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.parseDNFromHostsEntry(DatanodeManager.java:970)
           at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.getDatanodeListForReport(DatanodeManager.java:1039)
           at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.fetchDatanodes(DatanodeManager.java:892)
           at org.apache.hadoop.hdfs.server.namenode.NamenodeJspHelper$HealthJsp.generateHealthReport(NamenodeJspHelper.java:288)
           at org.apache.hadoop.hdfs.server.namenode.dfshealth_jsp._jspService(dfshealth_jsp.java:109)
           at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98)
           at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
           at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
           at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
           at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
           at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
           at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1071)
           at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
           at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
           at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
           at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
           at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
           at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
           at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
           at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
           at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
           at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
           at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
           at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
           at org.mortbay.jetty.Server.handle(Server.java:326)
           at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
           at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
           at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
           at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
           at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
           at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
           at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
      

      It's probably because DatanodeManager.parseDNFromHostsEntry() doesn't parse host:port string correctly.

        private DatanodeID parseDNFromHostsEntry(String hostLine) {
          DatanodeID dnId;
          String hostStr;
          int port;
          int idx = hostLine.indexOf(':');
      
          if (-1 == idx) {
            hostStr = hostLine;
            port = DFSConfigKeys.DFS_DATANODE_DEFAULT_PORT;
          } else {
            hostStr = hostLine.substring(0, idx);
            port = Integer.valueOf(hostLine.substring(idx)); // <- HERE!!
          }
      

      correct it as the below.

      port = Integer.valueOf(hostLine.substring(idx + 1));
      

      Attachments

        1. hdfs4344-1.txt
          8 kB
          Andy Isaacson
        2. hdfs4344.txt
          8 kB
          Andy Isaacson

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            adi2 Andy Isaacson
            tamtam180 tamtam180
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment