Hadoop Common
  1. Hadoop Common
  2. HADOOP-5200

NPE when the namenode comes up but the filesystem is set to file://

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Duplicate
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None

      Description

      If you bring up a namenode and the conf file points to file:/// as the URI, then the authority is null, breaking code that follows

        Issue Links

          Activity

          Steve Loughran created issue -
          Hide
          Steve Loughran added a comment -

          stack trace is in my patched code, but the problem lines in NameNode.getAddress(); the assumption that FileSystem.getDefaultUri(conf).getAuthority() is never null

          java.lang.NullPointerException
          at org.apache.hadoop.util.Service.deploy(Service.java:668)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1041)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1050)
          Caused by: java.lang.NullPointerException
          at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:134)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:159)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:163)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:179)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.innerStart(NameNode.java:300)
          at org.apache.hadoop.util.Service.start(Service.java:186)
          at org.apache.hadoop.util.Service.deploy(Service.java:655)
          ... 2 more

          Show
          Steve Loughran added a comment - stack trace is in my patched code, but the problem lines in NameNode.getAddress(); the assumption that FileSystem.getDefaultUri(conf).getAuthority() is never null java.lang.NullPointerException at org.apache.hadoop.util.Service.deploy(Service.java:668) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1041) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1050) Caused by: java.lang.NullPointerException at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:134) at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:159) at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:163) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:179) at org.apache.hadoop.hdfs.server.namenode.NameNode.innerStart(NameNode.java:300) at org.apache.hadoop.util.Service.start(Service.java:186) at org.apache.hadoop.util.Service.deploy(Service.java:655) ... 2 more
          Hide
          Steve Loughran added a comment -

          I propose patching this so that the operation fails with an IllegalArgumentException that includes the FS URI, so lets you diagnose the problem. Testing will be trivial.

          Show
          Steve Loughran added a comment - I propose patching this so that the operation fails with an IllegalArgumentException that includes the FS URI, so lets you diagnose the problem. Testing will be trivial.
          Steve Loughran made changes -
          Field Original Value New Value
          Assignee Steve Loughran [ steve_l ]
          Steve Loughran made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Hide
          Steve Loughran added a comment -

          the fix to NameNode for this is
          public static InetSocketAddress getAddress(Configuration conf) {
          URI fsURI = FileSystem.getDefaultUri(conf);
          if (fsURI == null)

          { throw new IllegalArgumentException( "No default filesystem URI in the configuration"); }

          String auth = fsURI.getAuthority();
          if (auth == null)

          { throw new IllegalArgumentException( "No authority for the Filesystem URI " + fsURI); }

          return getAddress(auth);
          }
          I've incorporated it into HADOOP-3628, but it is standalone and the three unit tests it needs wouldn't even need a namenode to be created.

          Show
          Steve Loughran added a comment - the fix to NameNode for this is public static InetSocketAddress getAddress(Configuration conf) { URI fsURI = FileSystem.getDefaultUri(conf); if (fsURI == null) { throw new IllegalArgumentException( "No default filesystem URI in the configuration"); } String auth = fsURI.getAuthority(); if (auth == null) { throw new IllegalArgumentException( "No authority for the Filesystem URI " + fsURI); } return getAddress(auth); } I've incorporated it into HADOOP-3628 , but it is standalone and the three unit tests it needs wouldn't even need a namenode to be created.
          Hide
          Philip Zeyliger added a comment -

          Steve,

          Is this similar to HADOOP-5687? We should get these committed to trunk; it's irritating that users see so many NPEs.

          – Philip

          Show
          Philip Zeyliger added a comment - Steve, Is this similar to HADOOP-5687 ? We should get these committed to trunk; it's irritating that users see so many NPEs. – Philip
          Hide
          Konstantin Shvachko added a comment -

          Is it the same as HADOOP-5687?
          This is about name-nodes not data-nodes, right? The title is confusing.

          Show
          Konstantin Shvachko added a comment - Is it the same as HADOOP-5687 ? This is about name-nodes not data-nodes, right? The title is confusing.
          Konstantin Shvachko made changes -
          Link This issue is related to HADOOP-5687 [ HADOOP-5687 ]
          Hide
          Steve Loughran added a comment -

          correcting the comment

          Show
          Steve Loughran added a comment - correcting the comment
          Steve Loughran made changes -
          Summary NPE when the datanode comes up but the filesystem is set to file:// NPE when the namenode comes up but the filesystem is set to file://
          Hide
          Steve Loughran added a comment -

          Yes, this duplicates HADOOP-5687, thanks for committing that

          -my code adds a check for FileSystem.getDefaultUri(conf) being null, that could go in on top of what is checked in. It's unlikely, but if you try hard enough, it happens.

          -some tests for this would be lightweight and fast. I will write them.

          Show
          Steve Loughran added a comment - Yes, this duplicates HADOOP-5687 , thanks for committing that -my code adds a check for FileSystem.getDefaultUri(conf) being null, that could go in on top of what is checked in. It's unlikely, but if you try hard enough, it happens. -some tests for this would be lightweight and fast. I will write them.
          Steve Loughran made changes -
          Link This issue duplicates HADOOP-5687 [ HADOOP-5687 ]
          Hide
          Steve Loughran added a comment -

          linking to the fixed issue

          Show
          Steve Loughran added a comment - linking to the fixed issue
          Steve Loughran made changes -
          Status In Progress [ 3 ] Resolved [ 5 ]
          Fix Version/s 0.21.0 [ 12313563 ]
          Resolution Duplicate [ 3 ]
          Steve Loughran made changes -
          Link This issue is related to HADOOP-5905 [ HADOOP-5905 ]
          Owen O'Malley made changes -
          Component/s dfs [ 12310710 ]
          Gavin made changes -
          Reporter Steve Loughran [ steve_l ] Steve Loughran [ stevel@apache.org ]
          Gavin made changes -
          Assignee Steve Loughran [ steve_l ] Steve Loughran [ stevel@apache.org ]
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open In Progress In Progress
          101d 2h 57m 1 Steve Loughran 21/May/09 17:09
          In Progress In Progress Resolved Resolved
          21h 47m 1 Steve Loughran 22/May/09 14:57
          Resolved Resolved Closed Closed
          459d 6h 38m 1 Tom White 24/Aug/10 21:35

            People

            • Assignee:
              Steve Loughran
              Reporter:
              Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development