HBase
  1. HBase
  2. HBASE-3138

When new master joins running cluster but meta is yanked from it as processing RIT, gets unexpected state

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: None
    • Fix Version/s: 0.92.3
    • Component/s: None
    • Labels:
      None

      Description

      Testing rolling restart i turned up the following condition.

      Master is joining an extant cluster and is trying to clean up RIT. Then the server hosting .META. is shutdown in the middle of it all. Deal. Here is exception.

      2010-10-21 06:45:58,592 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_OPENED, server=sv2borg187,60020,1287643131919, region=efcd899283e96f20faa317772f52adca
      2010-10-21 06:45:58,616 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown.
      org.apache.hadoop.ipc.RemoteException: java.io.IOException: Server not running
          at org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:2198)
          at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1499)
          at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:597)
          at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:561)
          at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1025)
      
          at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:749)
          at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:255)
          at $Proxy1.get(Unknown Source)
          at org.apache.hadoop.hbase.catalog.MetaReader.getRegion(MetaReader.java:286)
          at org.apache.hadoop.hbase.master.AssignmentManager.processRegionInTransition(AssignmentManager.java:250)
          at org.apache.hadoop.hbase.master.AssignmentManager.processFailover(AssignmentManager.java:209)
          at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:392)
          at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:268)
      2010-10-21 06:45:58,617 INFO org.apache.hadoop.hbase.master.HMaster: Aborting
      

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Unassigned
              Reporter:
              stack
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development