Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      http://54.241.6.143/job/HBase-0.95/665/org.apache.hbase$hbase-server/testReport/org.apache.hadoop.hbase.regionserver.wal/TestHLog/testAppendClose/

      Error Message
      
      Problem binding to localhost/127.0.0.1:37036 : Address already in use
      Stacktrace
      
      java.net.BindException: Problem binding to localhost/127.0.0.1:37036 : Address already in use
      	at org.apache.hadoop.ipc.Server.bind(Server.java:228)
      	at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:302)
      	at org.apache.hadoop.ipc.Server.<init>(Server.java:1488)
      	at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:560)
      	at org.apache.hadoop.ipc.RPC.getServer(RPC.java:521)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:302)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:536)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1410)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:278)
      	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSClusterForTestHLog(HBaseTestingUtility.java:525)
      ...
      

      This testAppendClose stops hdfs and starts it again. It looks problematic. Has waits of 7 seconds for the hdfs cluster to go down but in this test it seems like it needs even more time.

        Activity

        stack created issue -
        Hide
        stack added a comment -

        I tried a few things. The big long 7 second wait seems pretty necessary otherwise I get:

         3 Tests run: 13, Failures: 0, Errors: 5, Skipped: 0, Time elapsed: 52.038 sec <<< FAILURE!
          4 testAppendClose(org.apache.hadoop.hbase.regionserver.wal.TestHLog)  Time elapsed: 40.121 sec  <<< ERROR!
          5 java.io.IOException: Cannot lock storage /Users/stack/checkouts/trunk/hbase-server/target/test-data/b7723583-fda7-46c7-a3b5-bde04f2f9b77/dfscluster_945339f9-1cd2-416f-a3e1-0e8a89a4e10a/dfs/name1. T#
          6 ,...at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.lock(Storage.java:599)
          7 ,...at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:452)
          8 ,...at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:298)
          9 ,...at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
         10 ,...at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:411)
         11 ,...at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:379)
         12 ,...at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:284)
         13 ,...at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:536)
         14 ,...at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1410)
        ...
        

        I am tempted to disable this test but it is kinda important.

        Leaving open for now to keep an eye on it. Any input appreciated.

        Show
        stack added a comment - I tried a few things. The big long 7 second wait seems pretty necessary otherwise I get: 3 Tests run: 13, Failures: 0, Errors: 5, Skipped: 0, Time elapsed: 52.038 sec <<< FAILURE! 4 testAppendClose(org.apache.hadoop.hbase.regionserver.wal.TestHLog) Time elapsed: 40.121 sec <<< ERROR! 5 java.io.IOException: Cannot lock storage /Users/stack/checkouts/trunk/hbase-server/target/test-data/b7723583-fda7-46c7-a3b5-bde04f2f9b77/dfscluster_945339f9-1cd2-416f-a3e1-0e8a89a4e10a/dfs/name1. T# 6 ,...at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.lock(Storage.java:599) 7 ,...at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:452) 8 ,...at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:298) 9 ,...at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100) 10 ,...at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:411) 11 ,...at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:379) 12 ,...at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:284) 13 ,...at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:536) 14 ,...at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1410) ... I am tempted to disable this test but it is kinda important. Leaving open for now to keep an eye on it. Any input appreciated.

          People

          • Assignee:
            Unassigned
            Reporter:
            stack
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development