Tue Oct 20 17:06:20 GMT 2009 Starting master on cache01.dev02.oneriot.com ulimit -n 32768 2009-10-20 17:06:21,568 INFO org.apache.hadoop.hbase.master.HMaster: vmName=Java HotSpot(TM) 64-Bit Server VM, vmVendor=Sun Microsystems Inc., vmVersion=10.0-b23 2009-10-20 17:06:21,569 INFO org.apache.hadoop.hbase.master.HMaster: vmInputArguments=[-Xmx1000m, -XX:+HeapDumpOnOutOfMemoryError, -XX:+UseConcMarkSweepGC, -XX:+CMSIncrementalMode, -Dhbase.log.dir=/home/hadoop/hbase-0.20.0/bin/../logs, -Dhbase.log.file=hbase-hadoop-master-cache01.dev02.oneriot.com.log, -Dhbase.home.dir=/home/hadoop/hbase-0.20.0/bin/.., -Dhbase.id.str=hadoop, -Dhbase.root.logger=INFO,DRFA, -Djava.library.path=/home/hadoop/hbase-0.20.0/bin/../lib/native/Linux-amd64-64] 2009-10-20 17:06:21,657 INFO org.apache.hadoop.hbase.master.HMaster: My address is cache01.dev02.oneriot.com:59998 2009-10-20 17:06:22,171 INFO org.apache.hadoop.hbase.ipc.HBaseRpcMetrics: Initializing RPC Metrics with hostName=HMaster, port=59998 2009-10-20 17:06:22,349 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.2.0--1, built on 05/15/2009 06:05 GMT 2009-10-20 17:06:22,349 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=cache01.dev02.oneriot.com 2009-10-20 17:06:22,349 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.6.0_07 2009-10-20 17:06:22,349 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc. 2009-10-20 17:06:22,349 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/local/medium/jdk1.6.0_07/jre 2009-10-20 17:06:22,349 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=/home/hadoop/hbase-0.20.0/conf/:/usr/local/medium/jdk//lib/tools.jar:/home/hadoop/hbase-0.20.0/bin/..:/home/hadoop/hbase-0.20.0/bin/../hbase-0.20.0-ONERIOT.jar:/home/hadoop/hbase-0.20.0/bin/../lib/AgileJSON-2009-03-30.jar:/home/hadoop/hbase-0.20.0/bin/../lib/commons-cli-2.0-SNAPSHOT.jar:/home/hadoop/hbase-0.20.0/bin/../lib/commons-el-from-jetty-5.1.4.jar:/home/hadoop/hbase-0.20.0/bin/../lib/commons-httpclient-3.0.1.jar:/home/hadoop/hbase-0.20.0/bin/../lib/commons-logging-1.0.4.jar:/home/hadoop/hbase-0.20.0/bin/../lib/commons-logging-api-1.0.4.jar:/home/hadoop/hbase-0.20.0/bin/../lib/commons-math-1.1.jar:/home/hadoop/hbase-0.20.0/bin/../lib/hadoop-0.20.0-plus4681-core.jar:/home/hadoop/hbase-0.20.0/bin/../lib/hadoop-0.20.0-test.jar:/home/hadoop/hbase-0.20.0/bin/../lib/jasper-compiler-5.5.12.jar:/home/hadoop/hbase-0.20.0/bin/../lib/jasper-runtime-5.5.12.jar:/home/hadoop/hbase-0.20.0/bin/../lib/jetty-6.1.14.jar:/home/hadoop/hbase-0.20.0/bin/../lib/jetty-util-6.1.14.jar:/home/hadoop/hbase-0.20.0/bin/../lib/jruby-complete-1.2.0.jar:/home/hadoop/hbase-0.20.0/bin/../lib/json.jar:/home/hadoop/hbase-0.20.0/bin/../lib/junit-3.8.1.jar:/home/hadoop/hbase-0.20.0/bin/../lib/libthrift-r771587.jar:/home/hadoop/hbase-0.20.0/bin/../lib/log4j-1.2.15.jar:/home/hadoop/hbase-0.20.0/bin/../lib/lucene-core-2.2.0.jar:/home/hadoop/hbase-0.20.0/bin/../lib/servlet-api-2.5-6.1.14.jar:/home/hadoop/hbase-0.20.0/bin/../lib/xmlenc-0.52.jar:/home/hadoop/hbase-0.20.0/bin/../lib/zookeeper-r785019-hbase-1329.jar:/home/hadoop/hbase-0.20.0/bin/../lib/jsp-2.1/jsp-2.1.jar:/home/hadoop/hbase-0.20.0/bin/../lib/jsp-2.1/jsp-api-2.1.jar 2009-10-20 17:06:22,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/home/hadoop/hbase-0.20.0/bin/../lib/native/Linux-amd64-64 2009-10-20 17:06:22,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 2009-10-20 17:06:22,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler= 2009-10-20 17:06:22,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux 2009-10-20 17:06:22,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64 2009-10-20 17:06:22,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=2.6.18-128.1.10.el5 2009-10-20 17:06:22,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=hadoop 2009-10-20 17:06:22,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/home/hadoop 2009-10-20 17:06:22,350 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/home/hadoop/hbase-0.20.0/bin 2009-10-20 17:06:22,352 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, host=search01.dev02.oneriot.com:2181,cache02.dev02.oneriot.com:2181,cache01.dev02.oneriot.com:2181 sessionTimeout=60000 watcher=Thread[Thread-0,5,main] 2009-10-20 17:06:22,354 INFO org.apache.zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset is false 2009-10-20 17:06:22,365 INFO org.apache.zookeeper.ClientCnxn: Attempting connection to server search01.dev02.oneriot.com/10.129.1.186:2181 2009-10-20 17:06:22,367 INFO org.apache.zookeeper.ClientCnxn: Priming connection to java.nio.channels.SocketChannel[connected local=/10.129.1.181:35211 remote=search01.dev02.oneriot.com/10.129.1.186:2181] 2009-10-20 17:06:22,383 INFO org.apache.zookeeper.ClientCnxn: Server connection successful 2009-10-20 17:06:22,396 INFO org.apache.hadoop.hbase.master.RegionManager: -ROOT- region unset (but not set to be reassigned) 2009-10-20 17:06:22,397 INFO org.apache.hadoop.hbase.master.RegionManager: ROOT inserted into regionsInTransition 2009-10-20 17:07:22,389 INFO org.apache.hadoop.hbase.master.ServerManager: 0 region servers, 0 dead, average load NaN 2009-10-20 17:07:55,262 WARN org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to set state node in ZooKeeper org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /hbase/shutdown at org.apache.zookeeper.KeeperException.create(KeeperException.java:110) at org.apache.zookeeper.KeeperException.create(KeeperException.java:42) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:522) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.setClusterState(ZooKeeperWrapper.java:279) at org.apache.hadoop.hbase.master.HMaster.writeAddressToZooKeeper(HMaster.java:270) at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:255) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1200) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1241) 2009-10-20 17:07:55,265 INFO org.apache.hadoop.hbase.master.HMaster: HMaster initialized on 10.129.1.181:59998 2009-10-20 17:07:55,275 INFO org.apache.hadoop.hbase.master.HMaster: This is a failover, ZK inspection begins... 2009-10-20 17:07:55,513 INFO org.apache.hadoop.hbase.master.HMaster: Inspection found 159 regions, with -ROOT- 2009-10-20 17:07:55,518 INFO org.apache.hadoop.hbase.master.HMaster: Found log folder : cache01.dev02.oneriot.com,60020,1255390684310 2009-10-20 17:07:55,518 INFO org.apache.hadoop.hbase.master.HMaster: Log folder doesn't belong to a known region server, splitting 2009-10-20 17:07:55,522 INFO org.apache.hadoop.hbase.master.HMaster: Found log folder : search01.dev02.oneriot.com,60020,1256055534155 2009-10-20 17:07:55,522 INFO org.apache.hadoop.hbase.master.HMaster: Log folder doesn't belong to a known region server, splitting 2009-10-20 17:07:55,527 INFO org.apache.hadoop.hbase.regionserver.HLog: Splitting 1 hlog(s) in hdfs://cache01.dev02.oneriot.com:9000/hbase/.logs/search01.dev02.oneriot.com,60020,1256055534155 2009-10-20 17:07:55,541 WARN org.apache.hadoop.hbase.regionserver.HLog: Empty hlog, continuing: org.apache.hadoop.fs.FileStatus@679c6445 count=0 java.io.EOFException at java.io.DataInputStream.readFully(DataInputStream.java:180) at java.io.DataInputStream.readFully(DataInputStream.java:152) at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1450) at org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1428) at org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1417) at org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1412) at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:876) at org.apache.hadoop.hbase.regionserver.HLog.splitLog(HLog.java:802) at org.apache.hadoop.hbase.master.HMaster.splitLogAfterStartup(HMaster.java:614) at org.apache.hadoop.hbase.master.HMaster.verifyClusterState(HMaster.java:585) at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:410) 2009-10-20 17:07:55,549 INFO org.apache.hadoop.hbase.regionserver.HLog: hlog file splitting completed in 26 millis for hdfs://cache01.dev02.oneriot.com:9000/hbase/.logs/search01.dev02.oneriot.com,60020,1256055534155 2009-10-20 17:07:55,553 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=Master, sessionId=HMaster 2009-10-20 17:07:55,554 INFO org.apache.hadoop.hbase.master.metrics.MasterMetrics: Initialized 2009-10-20 17:07:55,556 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner scanning meta region {server: 10.129.1.181:60020, regionname: -ROOT-,,0, startKey: <>} 2009-10-20 17:07:55,556 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.metaScanner scanning meta region {server: 10.129.1.181:60020, regionname: .META.,,1, startKey: <>} 2009-10-20 17:07:55,627 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner scan of 1 row(s) of meta region {server: 10.129.1.181:60020, regionname: -ROOT-,,0, startKey: <>} complete 2009-10-20 17:07:55,822 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 60010 2009-10-20 17:07:55,961 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.metaScanner scan of 158 row(s) of meta region {server: 10.129.1.181:60020, regionname: .META.,,1, startKey: <>} complete 2009-10-20 17:07:56,490 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server Responder: starting 2009-10-20 17:07:56,490 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 59998: starting 2009-10-20 17:07:56,493 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 59998: starting 2009-10-20 17:07:56,493 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 2 on 59998: starting 2009-10-20 17:07:56,493 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 59998: starting 2009-10-20 17:07:56,494 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 59998: starting 2009-10-20 17:07:56,494 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 4 on 59998: starting 2009-10-20 17:07:56,495 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 5 on 59998: starting 2009-10-20 17:07:56,495 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 6 on 59998: starting 2009-10-20 17:07:56,495 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 7 on 59998: starting 2009-10-20 17:07:56,496 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 8 on 59998: starting 2009-10-20 17:07:56,496 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 9 on 59998: starting 2009-10-20 17:07:57,960 INFO org.apache.hadoop.hbase.master.RegionManager: exiting safe mode 2009-10-20 17:08:06,293 INFO org.apache.hadoop.hbase.master.ServerManager: cache01.dev02.oneriot.com,60020,1255975651472 znode expired 2009-10-20 17:08:06,298 INFO org.apache.hadoop.hbase.master.RegionManager: -ROOT- region unset (but not set to be reassigned) 2009-10-20 17:08:06,298 INFO org.apache.hadoop.hbase.master.RegionManager: META region removed from onlineMetaRegions 2009-10-20 17:08:06,298 INFO org.apache.hadoop.hbase.master.RegionServerOperation: process shutdown of server cache01.dev02.oneriot.com,60020,1255975651472: logSplit: false, rootRescanned: false, numberOfMetaRegions: 1, onlineMetaRegions.size(): 0 2009-10-20 17:08:06,302 INFO org.apache.hadoop.hbase.master.RegionServerOperation: Log split complete, meta reassignment and scanning: 2009-10-20 17:08:06,302 INFO org.apache.hadoop.hbase.master.RegionServerOperation: ProcessServerShutdown reassigning ROOT region 2009-10-20 17:08:06,302 INFO org.apache.hadoop.hbase.master.RegionManager: -ROOT- region unset (but not set to be reassigned) 2009-10-20 17:08:06,302 INFO org.apache.hadoop.hbase.master.RegionManager: ROOT inserted into regionsInTransition 2009-10-20 17:08:06,302 INFO org.apache.hadoop.hbase.master.RegionServerOperation: ProcessServerShutdown setting to unassigned: {server: 10.129.1.181:60020, regionname: .META.,,1, startKey: <>} 2009-10-20 17:08:07,017 INFO org.apache.hadoop.hbase.master.RegionManager: Assigning ROOT 2009-10-20 17:08:07,017 INFO org.apache.hadoop.hbase.master.RegionManager: singleServer:false 2009-10-20 17:08:07,021 INFO org.apache.hadoop.hbase.master.RegionManager: Assigning region -ROOT-,,0 to 10.129.1.186,60020,1256055534155 2009-10-20 17:08:09,213 INFO org.apache.hadoop.hbase.master.RegionManager: Assigning region .META.,,1 to 10.129.1.182,60020,1256055520818 2009-10-20 17:08:10,043 INFO org.apache.hadoop.hbase.master.ServerManager: Processing MSG_REPORT_PROCESS_OPEN: -ROOT-,,0 from 10.129.1.186,60020,1256055534155; 1 of 2 2009-10-20 17:08:10,044 INFO org.apache.hadoop.hbase.master.ServerManager: Processing MSG_REPORT_OPEN: -ROOT-,,0 from 10.129.1.186,60020,1256055534155; 2 of 2 2009-10-20 17:08:19,276 INFO org.apache.hadoop.hbase.master.ServerManager: 10.129.1.182,60020,1256055520818 znode expired 2009-10-20 17:08:19,278 INFO org.apache.hadoop.hbase.master.RegionServerOperation: process shutdown of server 10.129.1.182,60020,1256055520818: logSplit: false, rootRescanned: false, numberOfMetaRegions: 1, onlineMetaRegions.size(): 0 2009-10-20 17:08:19,282 INFO org.apache.hadoop.hbase.master.RegionServerOperation: Log split complete, meta reassignment and scanning: 2009-10-20 17:08:22,400 INFO org.apache.hadoop.hbase.master.ServerManager: 1 region servers, 2 dead, average load 56.0[cache01.dev02.oneriot.com,60020,1255975651472, 10.129.1.182,60020,1256055520818] 2009-10-20 17:08:24,562 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, host=search01.dev02.oneriot.com:2181,cache02.dev02.oneriot.com:2181,cache01.dev02.oneriot.com:2181 sessionTimeout=60000 watcher=org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher@8dc1f04 2009-10-20 17:08:24,563 INFO org.apache.zookeeper.ClientCnxn: Attempting connection to server cache02.dev02.oneriot.com/10.129.1.182:2181 2009-10-20 17:08:24,564 INFO org.apache.zookeeper.ClientCnxn: Priming connection to java.nio.channels.SocketChannel[connected local=/10.129.1.181:34704 remote=cache02.dev02.oneriot.com/10.129.1.182:2181] 2009-10-20 17:08:24,564 INFO org.apache.zookeeper.ClientCnxn: Server connection successful 2009-10-20 17:08:25,605 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 0 time(s). 2009-10-20 17:08:26,608 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 1 time(s). 2009-10-20 17:08:27,611 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 2 time(s). 2009-10-20 17:08:28,615 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 3 time(s). 2009-10-20 17:08:29,618 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 4 time(s). 2009-10-20 17:08:30,621 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 5 time(s). 2009-10-20 17:08:31,623 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 6 time(s). 2009-10-20 17:08:32,625 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 7 time(s). 2009-10-20 17:08:33,627 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 8 time(s). 2009-10-20 17:08:34,629 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 9 time(s). 2009-10-20 17:08:37,643 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 0 time(s). 2009-10-20 17:08:38,646 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 1 time(s). 2009-10-20 17:08:39,649 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 2 time(s). 2009-10-20 17:08:40,653 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 3 time(s). 2009-10-20 17:08:41,656 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 4 time(s). 2009-10-20 17:08:42,659 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 5 time(s). 2009-10-20 17:08:43,661 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 6 time(s). 2009-10-20 17:08:44,663 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 7 time(s). 2009-10-20 17:08:45,665 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 8 time(s). 2009-10-20 17:08:46,667 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 9 time(s). 2009-10-20 17:08:49,679 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 0 time(s). 2009-10-20 17:08:50,682 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 1 time(s). 2009-10-20 17:08:51,684 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 2 time(s). 2009-10-20 17:08:52,687 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 3 time(s). 2009-10-20 17:08:53,690 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 4 time(s). 2009-10-20 17:08:54,693 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 5 time(s). 2009-10-20 17:08:55,566 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner scanning meta region {server: 10.129.1.186:60020, regionname: -ROOT-,,0, startKey: <>} 2009-10-20 17:08:55,567 INFO org.apache.hadoop.hbase.master.BaseScanner: All 0 .META. region(s) scanned 2009-10-20 17:08:55,575 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner scan of 1 row(s) of meta region {server: 10.129.1.186:60020, regionname: -ROOT-,,0, startKey: <>} complete 2009-10-20 17:08:55,696 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 6 time(s). 2009-10-20 17:08:56,699 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 7 time(s). 2009-10-20 17:08:57,702 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 8 time(s). 2009-10-20 17:08:58,705 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 9 time(s). 2009-10-20 17:09:09,344 INFO org.apache.hadoop.hbase.master.RegionServerOperation: process shutdown of server cache01.dev02.oneriot.com,60020,1255975651472: logSplit: true, rootRescanned: false, numberOfMetaRegions: 1, onlineMetaRegions.size(): 0 2009-10-20 17:09:09,344 INFO org.apache.hadoop.hbase.master.RegionServerOperation: Log split complete, meta reassignment and scanning: 2009-10-20 17:09:10,729 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 0 time(s). 2009-10-20 17:09:11,732 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 1 time(s). 2009-10-20 17:09:12,735 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 2 time(s). 2009-10-20 17:09:13,738 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 3 time(s). 2009-10-20 17:09:14,740 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 4 time(s). 2009-10-20 17:09:15,742 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 5 time(s). 2009-10-20 17:09:16,744 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 6 time(s). 2009-10-20 17:09:17,746 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 7 time(s). 2009-10-20 17:09:18,748 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 8 time(s). 2009-10-20 17:09:19,357 INFO org.apache.hadoop.hbase.master.RegionServerOperation: process shutdown of server 10.129.1.182,60020,1256055520818: logSplit: true, rootRescanned: true, numberOfMetaRegions: 1, onlineMetaRegions.size(): 0 2009-10-20 17:09:19,357 INFO org.apache.hadoop.hbase.master.RegionServerOperation: Log split complete, meta reassignment and scanning: 2009-10-20 17:09:19,750 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 9 time(s). 2009-10-20 17:09:22,411 INFO org.apache.hadoop.hbase.master.ServerManager: 1 region servers, 2 dead, average load 56.0[cache01.dev02.oneriot.com,60020,1255975651472, 10.129.1.182,60020,1256055520818] 2009-10-20 17:09:22,762 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 0 time(s). 2009-10-20 17:09:23,765 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 1 time(s). 2009-10-20 17:09:24,768 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 2 time(s). 2009-10-20 17:09:25,771 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 3 time(s). 2009-10-20 17:09:26,773 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 4 time(s). 2009-10-20 17:09:27,775 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 5 time(s). 2009-10-20 17:09:28,777 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 6 time(s). 2009-10-20 17:09:29,780 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 7 time(s). 2009-10-20 17:09:30,783 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 8 time(s). 2009-10-20 17:09:31,786 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 9 time(s). 2009-10-20 17:09:34,798 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 0 time(s). 2009-10-20 17:09:35,801 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 1 time(s). 2009-10-20 17:09:36,804 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 2 time(s). 2009-10-20 17:09:37,807 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 3 time(s). 2009-10-20 17:09:38,810 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 4 time(s). 2009-10-20 17:09:39,812 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 5 time(s). 2009-10-20 17:09:40,814 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 6 time(s). 2009-10-20 17:09:41,817 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 7 time(s). 2009-10-20 17:09:42,820 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 8 time(s). 2009-10-20 17:09:43,822 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 9 time(s). 2009-10-20 17:09:55,577 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner scanning meta region {server: 10.129.1.186:60020, regionname: -ROOT-,,0, startKey: <>} 2009-10-20 17:09:55,578 INFO org.apache.hadoop.hbase.master.BaseScanner: All 0 .META. region(s) scanned 2009-10-20 17:09:55,586 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner scan of 1 row(s) of meta region {server: 10.129.1.186:60020, regionname: -ROOT-,,0, startKey: <>} complete 2009-10-20 17:10:09,372 INFO org.apache.hadoop.hbase.master.RegionServerOperation: process shutdown of server cache01.dev02.oneriot.com,60020,1255975651472: logSplit: true, rootRescanned: true, numberOfMetaRegions: 1, onlineMetaRegions.size(): 0 2009-10-20 17:10:09,372 INFO org.apache.hadoop.hbase.master.RegionServerOperation: Log split complete, meta reassignment and scanning: 2009-10-20 17:10:19,375 INFO org.apache.hadoop.hbase.master.RegionServerOperation: process shutdown of server 10.129.1.182,60020,1256055520818: logSplit: true, rootRescanned: true, numberOfMetaRegions: 1, onlineMetaRegions.size(): 0 2009-10-20 17:10:19,376 INFO org.apache.hadoop.hbase.master.RegionServerOperation: Log split complete, meta reassignment and scanning: 2009-10-20 17:10:22,421 INFO org.apache.hadoop.hbase.master.ServerManager: 1 region servers, 2 dead, average load 56.0[cache01.dev02.oneriot.com,60020,1255975651472, 10.129.1.182,60020,1256055520818] 2009-10-20 17:10:55,587 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner scanning meta region {server: 10.129.1.186:60020, regionname: -ROOT-,,0, startKey: <>} 2009-10-20 17:10:55,589 INFO org.apache.hadoop.hbase.master.BaseScanner: All 0 .META. region(s) scanned 2009-10-20 17:10:55,596 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner scan of 1 row(s) of meta region {server: 10.129.1.186:60020, regionname: -ROOT-,,0, startKey: <>} complete 2009-10-20 17:11:09,390 INFO org.apache.hadoop.hbase.master.RegionServerOperation: process shutdown of server cache01.dev02.oneriot.com,60020,1255975651472: logSplit: true, rootRescanned: true, numberOfMetaRegions: 1, onlineMetaRegions.size(): 0 2009-10-20 17:11:09,391 INFO org.apache.hadoop.hbase.master.RegionServerOperation: Log split complete, meta reassignment and scanning: 2009-10-20 17:11:13,232 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 0 time(s). 2009-10-20 17:11:14,235 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 1 time(s). 2009-10-20 17:11:15,238 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 2 time(s). 2009-10-20 17:11:16,241 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 3 time(s). 2009-10-20 17:11:17,244 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 4 time(s). 2009-10-20 17:11:18,248 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 5 time(s). 2009-10-20 17:11:19,251 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 6 time(s). 2009-10-20 17:11:19,394 INFO org.apache.hadoop.hbase.master.RegionServerOperation: process shutdown of server 10.129.1.182,60020,1256055520818: logSplit: true, rootRescanned: true, numberOfMetaRegions: 1, onlineMetaRegions.size(): 0 2009-10-20 17:11:19,394 INFO org.apache.hadoop.hbase.master.RegionServerOperation: Log split complete, meta reassignment and scanning: 2009-10-20 17:11:20,254 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 7 time(s). 2009-10-20 17:11:21,256 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 8 time(s). 2009-10-20 17:11:22,258 INFO org.apache.hadoop.ipc.HBaseClient: Retrying connect to server: /10.129.1.182:60020. Already tried 9 time(s). 2009-10-20 17:11:22,431 INFO org.apache.hadoop.hbase.master.ServerManager: 1 region servers, 2 dead, average load 56.0[cache01.dev02.oneriot.com,60020,1255975651472, 10.129.1.182,60020,1256055520818]