Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-3344

Master aborts after RPC to server that was shutting down

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.90.0
    • Fix Version/s: 0.90.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I was doing a rolling restart during a bunch of splits happening, and the master aborted with the following:

      2010-12-13 12:24:55,536 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Starting unassignment of region usertable,user1590589031,1291843166306.dbcbe21b3447c78560802962b87fd34f. (offlining)
      2010-12-13 12:24:55,537 FATAL org.apache.hadoop.hbase.master.HMaster: Remote unexpected exception
      java.io.IOException: Call to haus03.sf.cloudera.com/172.29.5.34:60020 failed on local exception: java.io.EOFException
      at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:788)
      at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:757)
      at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
      at $Proxy6.closeRegion(Unknown Source)
      at org.apache.hadoop.hbase.master.ServerManager.sendRegionClose(ServerManager.java:589)
      at org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1085)
      at org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1032)
      at org.apache.hadoop.hbase.master.AssignmentManager.balance(AssignmentManager.java:1791)
      at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:688)
      at org.apache.hadoop.hbase.master.HMaster$1.chore(HMaster.java:579)
      at org.apache.hadoop.hbase.Chore.run(Chore.java:66)
      Caused by: java.io.EOFException
      at java.io.DataInputStream.readInt(DataInputStream.java:375)
      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:521)
      at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:459)
      2010-12-13 12:24:55,541 INFO org.apache.hadoop.hbase.master.HMaster: Aborting

        Attachments

        1. 3344.txt
          0.8 kB
          Michael Stack

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              tlipcon Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: