Hadoop Common
  1. Hadoop Common
  2. HADOOP-9095

TestNNThroughputBenchmark fails in branch-1

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0, 1-win
    • Component/s: net
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      java.lang.StringIndexOutOfBoundsException: String index out of range: 0
          at java.lang.String.charAt(String.java:686)
          at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:539)
          at org.apache.hadoop.net.NetUtils.normalizeHostNames(NetUtils.java:562)
          at org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:88)
          at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1047)
          ...
          at org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark$StatsDaemon.run(NNThroughputBenchmark.java:377)
      
      1. HDFS-4204.b1.003.patch
        3 kB
        Jing Zhao
      2. HDFS-4204.b1.002.patch
        0.8 kB
        Jing Zhao
      3. HDFS-4204.b1.001.patch
        0.6 kB
        Jing Zhao

        Issue Links

          Activity

          Matt Foley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Matt Foley added a comment -

          Closed upon release of Hadoop 1.2.0.

          Show
          Matt Foley added a comment - Closed upon release of Hadoop 1.2.0.
          Tsz Wo Nicholas Sze made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s 1.2.0 [ 12321659 ]
          Fix Version/s 1-win [ 12320361 ]
          Resolution Fixed [ 1 ]
          Tsz Wo Nicholas Sze made changes -
          Link This issue relates to HADOOP-8372 [ HADOOP-8372 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed this. Thanks Jing!

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks Jing!
          Tsz Wo Nicholas Sze made changes -
          Hadoop Flags Reviewed [ 10343 ]
          Component/s net [ 12319327 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 patch looks good.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
          Tsz Wo Nicholas Sze made changes -
          Project Hadoop HDFS [ 12310942 ] Hadoop Common [ 12310240 ]
          Key HDFS-4204 HADOOP-9095
          Jing Zhao made changes -
          Attachment HDFS-4204.b1.003.patch [ 12554900 ]
          Hide
          Jing Zhao added a comment -

          The same test from HADOOP-8372 is also included.

          All the unit tests have passed except TestFileCreationNamenodeRestart, which should be unrelated since it also fails without the patch. The test-patch result:

          -1 overall.  
              +1 @author.  The patch does not contain any @author tags.
              +1 tests included.  The patch appears to include 3 new or modified tests.
              +1 javadoc.  The javadoc tool did not generate any warning messages.
              +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
              -1 findbugs.  The patch appears to introduce 225 new Findbugs (version 2.0.1) warnings.
          
          Show
          Jing Zhao added a comment - The same test from HADOOP-8372 is also included. All the unit tests have passed except TestFileCreationNamenodeRestart, which should be unrelated since it also fails without the patch. The test-patch result: -1 overall. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 225 new Findbugs (version 2.0.1) warnings.
          Jing Zhao made changes -
          Assignee Jing Zhao [ jingzhao ]
          Jing Zhao made changes -
          Priority Major [ 3 ] Minor [ 4 ]
          Jing Zhao made changes -
          Attachment HDFS-4204.b1.002.patch [ 12554890 ]
          Hide
          Jing Zhao added a comment -

          New patch uploaded. Will run ant test and testpatch for it.

          Show
          Jing Zhao added a comment - New patch uploaded. Will run ant test and testpatch for it.
          Hide
          Jing Zhao added a comment -

          The empty string passed here should be fine, since in trunk for the same test the string passed is also an empty string. In general the invocation of NetUtils#normalizeHostName with an empty string happens in this scenario: OpenFileStats#executeOp --> NameNode#getBlockLocations --> FSNamesystem#getBlockLocations --> CachedDNSToSwitchMapping#resolve --> NetUtils#normalizeHostNames --> NetUtils.normalizeHostName. Based on Junping's comments, I think we can simply backport HADOOP-8372 to fix the error.

          Show
          Jing Zhao added a comment - The empty string passed here should be fine, since in trunk for the same test the string passed is also an empty string. In general the invocation of NetUtils#normalizeHostName with an empty string happens in this scenario: OpenFileStats#executeOp --> NameNode#getBlockLocations --> FSNamesystem#getBlockLocations --> CachedDNSToSwitchMapping#resolve --> NetUtils#normalizeHostNames --> NetUtils.normalizeHostName. Based on Junping's comments, I think we can simply backport HADOOP-8372 to fix the error.
          Hide
          Junping Du added a comment -

          Shall we figure out why null or empty is passed here? Also, another bug is in judging IP or hostname (it is not right to check the first char only, as 05host.xy.z is also a possible hostname). HADOOP-8372 contains a fix for trunk, shall we backport to branch-1 as well?

          Show
          Junping Du added a comment - Shall we figure out why null or empty is passed here? Also, another bug is in judging IP or hostname (it is not right to check the first char only, as 05host.xy.z is also a possible hostname). HADOOP-8372 contains a fix for trunk, shall we backport to branch-1 as well?
          Hide
          Tsz Wo Nicholas Sze added a comment -
          +    if (name != null && !name.isEmpty()
          +        && Character.digit(name.charAt(0), 10) != -1) { // FIXME 
                 return name;
               } else {
          

          If name == null or name.isEmpty(), it does not make sense to resolve it using InetAddress. How about simply return name in such case?

          Show
          Tsz Wo Nicholas Sze added a comment - + if (name != null && !name.isEmpty() + && Character .digit(name.charAt(0), 10) != -1) { // FIXME return name; } else { If name == null or name.isEmpty(), it does not make sense to resolve it using InetAddress. How about simply return name in such case?
          Jing Zhao made changes -
          Field Original Value New Value
          Attachment HDFS-4204.b1.001.patch [ 12554555 ]
          Hide
          Jing Zhao added a comment -

          Seems a simple fix in NetUtils.normalizeHostName can make the testcase pass.

          Show
          Jing Zhao added a comment - Seems a simple fix in NetUtils.normalizeHostName can make the testcase pass.
          Tsz Wo Nicholas Sze created issue -

            People

            • Assignee:
              Jing Zhao
              Reporter:
              Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development