Hadoop Common
  1. Hadoop Common
  2. HADOOP-5200

NPE when the namenode comes up but the filesystem is set to file://

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Duplicate
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None

      Description

      If you bring up a namenode and the conf file points to file:/// as the URI, then the authority is null, breaking code that follows

        Issue Links

          Activity

          Hide
          steve_l added a comment -

          stack trace is in my patched code, but the problem lines in NameNode.getAddress(); the assumption that FileSystem.getDefaultUri(conf).getAuthority() is never null

          java.lang.NullPointerException
          at org.apache.hadoop.util.Service.deploy(Service.java:668)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1041)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1050)
          Caused by: java.lang.NullPointerException
          at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:134)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:159)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:163)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:179)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.innerStart(NameNode.java:300)
          at org.apache.hadoop.util.Service.start(Service.java:186)
          at org.apache.hadoop.util.Service.deploy(Service.java:655)
          ... 2 more

          Show
          steve_l added a comment - stack trace is in my patched code, but the problem lines in NameNode.getAddress(); the assumption that FileSystem.getDefaultUri(conf).getAuthority() is never null java.lang.NullPointerException at org.apache.hadoop.util.Service.deploy(Service.java:668) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1041) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1050) Caused by: java.lang.NullPointerException at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:134) at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:159) at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:163) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:179) at org.apache.hadoop.hdfs.server.namenode.NameNode.innerStart(NameNode.java:300) at org.apache.hadoop.util.Service.start(Service.java:186) at org.apache.hadoop.util.Service.deploy(Service.java:655) ... 2 more
          Hide
          steve_l added a comment -

          I propose patching this so that the operation fails with an IllegalArgumentException that includes the FS URI, so lets you diagnose the problem. Testing will be trivial.

          Show
          steve_l added a comment - I propose patching this so that the operation fails with an IllegalArgumentException that includes the FS URI, so lets you diagnose the problem. Testing will be trivial.
          Hide
          steve_l added a comment -

          the fix to NameNode for this is
          public static InetSocketAddress getAddress(Configuration conf) {
          URI fsURI = FileSystem.getDefaultUri(conf);
          if (fsURI == null)

          { throw new IllegalArgumentException( "No default filesystem URI in the configuration"); }

          String auth = fsURI.getAuthority();
          if (auth == null)

          { throw new IllegalArgumentException( "No authority for the Filesystem URI " + fsURI); }

          return getAddress(auth);
          }
          I've incorporated it into HADOOP-3628, but it is standalone and the three unit tests it needs wouldn't even need a namenode to be created.

          Show
          steve_l added a comment - the fix to NameNode for this is public static InetSocketAddress getAddress(Configuration conf) { URI fsURI = FileSystem.getDefaultUri(conf); if (fsURI == null) { throw new IllegalArgumentException( "No default filesystem URI in the configuration"); } String auth = fsURI.getAuthority(); if (auth == null) { throw new IllegalArgumentException( "No authority for the Filesystem URI " + fsURI); } return getAddress(auth); } I've incorporated it into HADOOP-3628 , but it is standalone and the three unit tests it needs wouldn't even need a namenode to be created.
          Hide
          Philip Zeyliger added a comment -

          Steve,

          Is this similar to HADOOP-5687? We should get these committed to trunk; it's irritating that users see so many NPEs.

          – Philip

          Show
          Philip Zeyliger added a comment - Steve, Is this similar to HADOOP-5687 ? We should get these committed to trunk; it's irritating that users see so many NPEs. – Philip
          Hide
          Konstantin Shvachko added a comment -

          Is it the same as HADOOP-5687?
          This is about name-nodes not data-nodes, right? The title is confusing.

          Show
          Konstantin Shvachko added a comment - Is it the same as HADOOP-5687 ? This is about name-nodes not data-nodes, right? The title is confusing.
          Hide
          steve_l added a comment -

          correcting the comment

          Show
          steve_l added a comment - correcting the comment
          Hide
          steve_l added a comment -

          Yes, this duplicates HADOOP-5687, thanks for committing that

          -my code adds a check for FileSystem.getDefaultUri(conf) being null, that could go in on top of what is checked in. It's unlikely, but if you try hard enough, it happens.

          -some tests for this would be lightweight and fast. I will write them.

          Show
          steve_l added a comment - Yes, this duplicates HADOOP-5687 , thanks for committing that -my code adds a check for FileSystem.getDefaultUri(conf) being null, that could go in on top of what is checked in. It's unlikely, but if you try hard enough, it happens. -some tests for this would be lightweight and fast. I will write them.
          Hide
          steve_l added a comment -

          linking to the fixed issue

          Show
          steve_l added a comment - linking to the fixed issue

            People

            • Assignee:
              Steve Loughran
              Reporter:
              Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development