ulimit -n 1024 2012-01-19 11:09:54,687 INFO org.apache.hadoop.hbase.ipc.HBaseRpcMetrics: Initializing RPC Metrics with hostName=HMaster, port=20000 2012-01-19 11:09:54,749 INFO org.apache.hadoop.hbase.security.User: Skipping login, not running on secure Hadoop 2012-01-19 11:09:54,751 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server Responder: starting 2012-01-19 11:09:54,751 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 20000: starting 2012-01-19 11:09:54,753 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 20000: starting 2012-01-19 11:09:54,753 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 20000: starting 2012-01-19 11:09:54,754 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 2 on 20000: starting 2012-01-19 11:09:54,754 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 20000: starting 2012-01-19 11:09:54,754 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 4 on 20000: starting 2012-01-19 11:09:54,755 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 5 on 20000: starting 2012-01-19 11:09:54,767 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 6 on 20000: starting 2012-01-19 11:09:54,768 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 7 on 20000: starting 2012-01-19 11:09:54,768 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 9 on 20000: starting 2012-01-19 11:09:54,769 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 8 on 20000: starting 2012-01-19 11:09:54,867 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32 GMT 2012-01-19 11:09:54,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=C3S31 2012-01-19 11:09:54,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.6.0_22 2012-01-19 11:09:54,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc. 2012-01-19 11:09:54,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=/opt/jdk1.6.0_22/jre 2012-01-19 11:09:54,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=/opt/hadoop/hbase/conf:/opt/jdk1.6.0_22/lib/tools.jar:/opt/hadoop/hbase:/opt/hadoop/hbase/hbase-0.90.6.jar:/opt/hadoop/hbase/hbase-core-tests.jar:/opt/hadoop/hbase/lib/activation-1.1.jar:/opt/hadoop/hbase/lib/asm-3.1.jar:/opt/hadoop/hbase/lib/avro-1.3.3.jar:/opt/hadoop/hbase/lib/commons-cli-1.2.jar:/opt/hadoop/hbase/lib/commons-codec-1.4.jar:/opt/hadoop/hbase/lib/commons-el-1.0.jar:/opt/hadoop/hbase/lib/commons-httpclient-3.1.jar:/opt/hadoop/hbase/lib/commons-lang-2.5.jar:/opt/hadoop/hbase/lib/commons-logging-1.1.1.jar:/opt/hadoop/hbase/lib/commons-net-1.4.1.jar:/opt/hadoop/hbase/lib/core-3.1.1.jar:/opt/hadoop/hbase/lib/guava-r06.jar:/opt/hadoop/hbase/lib/hadoop-core-0.20-append-r1057313.jar:/opt/hadoop/hbase/lib/jackson-core-asl-1.5.5.jar:/opt/hadoop/hbase/lib/jackson-jaxrs-1.5.5.jar:/opt/hadoop/hbase/lib/jackson-mapper-asl-1.4.2.jar:/opt/hadoop/hbase/lib/jackson-xc-1.5.5.jar:/opt/hadoop/hbase/lib/jasper-compiler-5.5.23.jar:/opt/hadoop/hbase/lib/jasper-runtime-5.5.23.jar:/opt/hadoop/hbase/lib/jaxb-api-2.1.jar:/opt/hadoop/hbase/lib/jaxb-impl-2.1.12.jar:/opt/hadoop/hbase/lib/jersey-core-1.4.jar:/opt/hadoop/hbase/lib/jersey-json-1.4.jar:/opt/hadoop/hbase/lib/jersey-server-1.4.jar:/opt/hadoop/hbase/lib/jettison-1.1.jar:/opt/hadoop/hbase/lib/jetty-6.1.26.jar:/opt/hadoop/hbase/lib/jetty-util-6.1.26.jar:/opt/hadoop/hbase/lib/jruby-complete-1.6.0.jar:/opt/hadoop/hbase/lib/jsp-2.1-6.1.14.jar:/opt/hadoop/hbase/lib/jsp-api-2.1-6.1.14.jar:/opt/hadoop/hbase/lib/jsr311-api-1.1.1.jar:/opt/hadoop/hbase/lib/log4j-1.2.16.jar:/opt/hadoop/hbase/lib/protobuf-java-2.3.0.jar:/opt/hadoop/hbase/lib/servlet-api-2.5-6.1.14.jar:/opt/hadoop/hbase/lib/slf4j-api-1.5.8.jar:/opt/hadoop/hbase/lib/slf4j-log4j12-1.5.8.jar:/opt/hadoop/hbase/lib/stax-api-1.0.1.jar:/opt/hadoop/hbase/lib/thrift-0.2.0.jar:/opt/hadoop/hbase/lib/xmlenc-0.52.jar:/opt/hadoop/hbase/lib/zookeeper.jar 2012-01-19 11:09:54,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/opt/jdk1.6.0_22/jre/lib/amd64/server:/opt/jdk1.6.0_22/jre/lib/amd64:/opt/jdk1.6.0_22/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2012-01-19 11:09:54,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 2012-01-19 11:09:54,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler= 2012-01-19 11:09:54,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux 2012-01-19 11:09:54,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64 2012-01-19 11:09:54,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=2.6.32.12-0.7-default 2012-01-19 11:09:54,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=root 2012-01-19 11:09:54,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/root 2012-01-19 11:09:54,868 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/opt/g56562 2012-01-19 11:09:54,869 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=C3S33:2181,C3S32:2181,C3S31:2181 sessionTimeout=300000 watcher=master:20000 2012-01-19 11:09:54,879 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server C3S33/158.1.130.33:2181 2012-01-19 11:09:54,882 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to C3S33/158.1.130.33:2181, initiating session 2012-01-19 11:09:54,892 INFO org.apache.jute.BinaryInputArchive: Expecting maxBufferString as an integer but got as null.Hence setting to default value 1048575 2012-01-19 11:09:54,894 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server C3S33/158.1.130.33:2181, sessionid = 0x334cb6fb0f0011e, negotiated timeout = 40000 2012-01-19 11:09:54,913 INFO org.apache.hadoop.hbase.metrics: MetricsString added: revision 2012-01-19 11:09:54,913 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsUser 2012-01-19 11:09:54,913 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsDate 2012-01-19 11:09:54,913 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsUrl 2012-01-19 11:09:54,913 INFO org.apache.hadoop.hbase.metrics: MetricsString added: date 2012-01-19 11:09:54,913 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsRevision 2012-01-19 11:09:54,913 INFO org.apache.hadoop.hbase.metrics: MetricsString added: user 2012-01-19 11:09:54,913 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsVersion 2012-01-19 11:09:54,913 INFO org.apache.hadoop.hbase.metrics: MetricsString added: url 2012-01-19 11:09:54,913 INFO org.apache.hadoop.hbase.metrics: MetricsString added: version 2012-01-19 11:09:54,913 INFO org.apache.hadoop.hbase.metrics: new MBeanInfo 2012-01-19 11:09:54,914 INFO org.apache.hadoop.hbase.metrics: new MBeanInfo 2012-01-19 11:09:54,914 INFO org.apache.hadoop.hbase.master.metrics.MasterMetrics: Initialized 2012-01-19 11:09:54,962 INFO org.apache.hadoop.hbase.master.ActiveMasterManager: Current master has this master's address, C3S31:20000; master was restarted? Waiting on znode to expire... 2012-01-19 11:09:54,986 DEBUG org.apache.hadoop.hbase.master.ActiveMasterManager: No master available. Notifying waiting threads 2012-01-19 11:09:55,021 INFO org.apache.hadoop.hbase.master.ActiveMasterManager: Master=C3S31:20000 2012-01-19 11:09:55,023 DEBUG org.apache.hadoop.hbase.master.ActiveMasterManager: A master is now available 2012-01-19 11:09:55,209 INFO com.huawei.isap.ump.ha.client.ProxyRPCConfigurationUtil: Following server values are configured for the property:namenode.servers values:[] 2012-01-19 11:09:55,210 WARN com.huawei.isap.ump.ha.client.ProxyRPCConfigurationUtil: Only 0 server(s) : [] configured for HA. Only the default address will be used 2012-01-19 11:09:55,211 INFO com.huawei.isap.ump.rpc.AbstractProxyFactory: DFSClient is in non-HA mode ... 2012-01-19 11:09:55,221 INFO org.apache.hadoop.ipc.Client: The value of 'ipc.client.connect.max.socket.retries' is : 45 2012-01-19 11:09:55,221 INFO org.apache.hadoop.ipc.Client: The value of 'max.ping.retries.on.socket.timeout' is :-1 2012-01-19 11:09:55,277 INFO org.apache.hadoop.hdfs.DFSClient: DFSClient has connected to the active namenode: C3S31/158.1.130.31:9000 2012-01-19 11:09:55,486 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=C3S33:2181,C3S32:2181,C3S31:2181 sessionTimeout=300000 watcher=hconnection 2012-01-19 11:09:55,488 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server C3S33/158.1.130.33:2181 2012-01-19 11:09:55,489 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to C3S33/158.1.130.33:2181, initiating session 2012-01-19 11:09:55,496 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server C3S33/158.1.130.33:2181, sessionid = 0x334cb6fb0f0011f, negotiated timeout = 40000 2012-01-19 11:09:55,514 DEBUG org.apache.hadoop.hbase.catalog.CatalogTracker: Starting catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@2de12f6d 2012-01-19 11:09:55,542 INFO org.apache.hadoop.hbase.master.HMaster: Server active/primary master; C3S31:20000, sessionid=0x334cb6fb0f0011e, cluster-up flag was=true 2012-01-19 11:09:55,544 DEBUG org.apache.hadoop.hbase.executor.ExecutorService: Starting executor service name=MASTER_OPEN_REGION-C3S31:20000, corePoolSize=8, maxPoolSize=8 2012-01-19 11:09:55,544 DEBUG org.apache.hadoop.hbase.executor.ExecutorService: Starting executor service name=MASTER_CLOSE_REGION-C3S31:20000, corePoolSize=5, maxPoolSize=5 2012-01-19 11:09:55,544 DEBUG org.apache.hadoop.hbase.executor.ExecutorService: Starting executor service name=MASTER_SERVER_OPERATIONS-C3S31:20000, corePoolSize=3, maxPoolSize=3 2012-01-19 11:09:55,544 DEBUG org.apache.hadoop.hbase.executor.ExecutorService: Starting executor service name=MASTER_META_SERVER_OPERATIONS-C3S31:20000, corePoolSize=5, maxPoolSize=5 2012-01-19 11:09:55,544 DEBUG org.apache.hadoop.hbase.executor.ExecutorService: Starting executor service name=MASTER_TABLE_OPERATIONS-C3S31:20000, corePoolSize=1, maxPoolSize=1 2012-01-19 11:09:55,546 DEBUG org.apache.hadoop.hbase.master.LogCleaner: Add log cleaner in chain: org.apache.hadoop.hbase.master.TimeToLiveLogCleaner 2012-01-19 11:09:55,623 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2012-01-19 11:09:55,677 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 20010 2012-01-19 11:09:55,678 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returned 20010 webServer.getConnectors()[0].getLocalPort() returned 20010 2012-01-19 11:09:55,678 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 20010 2012-01-19 11:09:55,678 INFO org.mortbay.log: jetty-6.1.26 2012-01-19 11:09:55,912 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:20010 2012-01-19 11:09:55,913 DEBUG org.apache.hadoop.hbase.master.HMaster: Started service threads 2012-01-19 11:09:57,413 INFO org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver(s) to checkin 2012-01-19 11:09:57,824 INFO org.apache.hadoop.hbase.master.ServerManager: Registering server=C3S31,20020,1326987900492, regionCount=1, userLoad=true 2012-01-19 11:09:58,591 INFO org.apache.hadoop.hbase.master.ServerManager: Registering server=C3S32,20020,1326987400439, regionCount=0, userLoad=true 2012-01-19 11:09:58,591 INFO org.apache.hadoop.hbase.master.ServerManager: Registering server=C3S33,20020,1326987399579, regionCount=1, userLoad=true 2012-01-19 11:09:58,914 INFO org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver(s) count to settle; currently=3 2012-01-19 11:10:00,414 INFO org.apache.hadoop.hbase.master.ServerManager: Finished waiting for regionserver count to settle; count=3, sleptFor=4500 2012-01-19 11:10:00,414 INFO org.apache.hadoop.hbase.master.ServerManager: Exiting wait on regionserver(s) to checkin; count=3, stopped=false, count of regions out on cluster=2 2012-01-19 11:10:00,414 INFO org.apache.hadoop.hbase.master.HMaster: ------------------sleep 60s----------------- 2012-01-19 11:10:35,847 INFO org.apache.hadoop.hbase.zookeeper.RegionServerTracker: RegionServer ephemeral node deleted, processing expiration [C3S31,20020,1326987900492] 2012-01-19 11:10:35,850 DEBUG org.apache.hadoop.hbase.master.ServerManager: Added=C3S31,20020,1326987900492 to dead servers, submitted shutdown handler to be executed, root=true, meta=false 2012-01-19 11:10:35,851 INFO org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Splitting logs for C3S31,20020,1326987900492 2012-01-19 11:10:35,851 INFO org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: ------------------ shuthander sleep 120s----------------- 2012-01-19 11:10:49,847 INFO org.apache.hadoop.hbase.zookeeper.RegionServerTracker: RegionServer ephemeral node deleted, processing expiration [C3S33,20020,1326987399579] 2012-01-19 11:10:49,848 DEBUG org.apache.hadoop.hbase.master.ServerManager: Added=C3S33,20020,1326987399579 to dead servers, submitted shutdown handler to be executed, root=false, meta=false 2012-01-19 11:10:49,849 INFO org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Splitting logs for C3S33,20020,1326987399579 2012-01-19 11:10:49,849 INFO org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: ------------------ shuthander sleep 120s----------------- 2012-01-19 11:11:00,423 INFO org.apache.hadoop.hbase.master.MasterFileSystem: Log folder hdfs://C3S31:9000/hbase/.logs/C3S31,20020,1326987900492 belongs to an existing region server 2012-01-19 11:11:00,423 INFO org.apache.hadoop.hbase.master.MasterFileSystem: Log folder hdfs://C3S31:9000/hbase/.logs/C3S32,20020,1326987400439 belongs to an existing region server 2012-01-19 11:11:00,424 INFO org.apache.hadoop.hbase.master.MasterFileSystem: Log folder hdfs://C3S31:9000/hbase/.logs/C3S33,20020,1326987399579 belongs to an existing region server 2012-01-19 11:11:00,441 INFO org.apache.hadoop.ipc.HbaseRPC: Server at C3S31/158.1.130.31:20020 could not be reached after 1 tries, giving up. 2012-01-19 11:12:35,871 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Splitting 1 hlog(s) in hdfs://C3S31:9000/hbase/.logs/C3S31,20020,1326987900492 2012-01-19 11:12:35,872 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Writer thread Thread[WriterThread-0,5,main]: starting 2012-01-19 11:12:35,873 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Writer thread Thread[WriterThread-1,5,main]: starting 2012-01-19 11:12:35,873 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Writer thread Thread[WriterThread-2,5,main]: starting 2012-01-19 11:12:35,877 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Writer thread Thread[WriterThread-3,5,main]: starting 2012-01-19 11:12:35,878 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Writer thread Thread[WriterThread-5,5,main]: starting 2012-01-19 11:12:35,877 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Writer thread Thread[WriterThread-4,5,main]: starting 2012-01-19 11:12:35,877 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Splitting hlog 1 of 1: hdfs://C3S31:9000/hbase/.logs/C3S31,20020,1326987900492/C3S31%3A20020.1326987901100, length=0 2012-01-19 11:12:35,880 INFO org.apache.hadoop.hbase.util.FSUtils: Recovering file hdfs://C3S31:9000/hbase/.logs/C3S31,20020,1326987900492/C3S31%3A20020.1326987901100 2012-01-19 11:12:36,884 INFO org.apache.hadoop.hbase.util.FSUtils: Finished lease recover attempt for hdfs://C3S31:9000/hbase/.logs/C3S31,20020,1326987900492/C3S31%3A20020.1326987901100 2012-01-19 11:12:36,884 WARN org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: File hdfs://C3S31:9000/hbase/.logs/C3S31,20020,1326987900492/C3S31%3A20020.1326987901100 might be still open, length is 0 2012-01-19 11:12:36,922 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Pushed=1 entries from hdfs://C3S31:9000/hbase/.logs/C3S31,20020,1326987900492/C3S31%3A20020.1326987901100 2012-01-19 11:12:36,931 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Waiting for split writer threads to finish 2012-01-19 11:12:36,950 INFO org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter: Using syncFs -- HDFS-200 2012-01-19 11:12:36,950 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Creating writer path=hdfs://C3S31:9000/hbase/-ROOT-/70236052/recovered.edits/0000000000000000004.temp region=70236052 2012-01-19 11:12:36,951 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Split writers finished 2012-01-19 11:12:36,994 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed path hdfs://C3S31:9000/hbase/-ROOT-/70236052/recovered.edits/0000000000000000004.temp (wrote 1 edits in 28ms) 2012-01-19 11:12:37,003 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://C3S31:9000/hbase/.logs/C3S31,20020,1326987900492/C3S31%3A20020.1326987901100 to hdfs://C3S31:9000/hbase/.oldlogs/C3S31%3A20020.1326987901100 2012-01-19 11:12:37,004 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: hlog file splitting completed in 1137 ms for hdfs://C3S31:9000/hbase/.logs/C3S31,20020,1326987900492 2012-01-19 11:12:37,004 INFO org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Server C3S31,20020,1326987900492 was carrying ROOT. Trying to assign. 2012-01-19 11:12:37,008 INFO org.apache.hadoop.ipc.HbaseRPC: Server at C3S31/158.1.130.31:20020 could not be reached after 1 tries, giving up. 2012-01-19 11:12:37,008 INFO org.apache.hadoop.hbase.catalog.RootLocationEditor: Unsetting ROOT region location in ZooKeeper 2012-01-19 11:12:37,015 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:20000-0x334cb6fb0f0011e Creating (or updating) unassigned node for 70236052 with OFFLINE state 2012-01-19 11:12:37,028 INFO org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan was found (or we are ignoring an existing plan) for -ROOT-,,0.70236052 so generated a random one; hri=-ROOT-,,0.70236052, src=, dest=C3S32,20020,1326987400439; 1 (online=1, exclude=null) available servers 2012-01-19 11:12:37,028 INFO org.apache.hadoop.hbase.master.AssignmentManager: Assigning region -ROOT-,,0.70236052 to C3S32,20020,1326987400439 2012-01-19 11:12:37,028 DEBUG org.apache.hadoop.hbase.master.ServerManager: New connection to C3S32,20020,1326987400439 2012-01-19 11:12:37,059 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_OPENING, server=C3S32,20020,1326987400439, region=70236052/-ROOT- 2012-01-19 11:12:37,218 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_OPENING, server=C3S32,20020,1326987400439, region=70236052/-ROOT- 2012-01-19 11:12:37,238 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_OPENED, server=C3S32,20020,1326987400439, region=70236052/-ROOT- 2012-01-19 11:12:37,240 INFO org.apache.hadoop.hbase.master.handler.OpenedRegionHandler: Handling OPENED event for -ROOT-,,0.70236052 from serverName=C3S32,20020,1326987400439, load=(requests=0, regions=0, usedHeap=32, maxHeap=991); deleting unassigned node 2012-01-19 11:12:37,240 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:20000-0x334cb6fb0f0011e Deleting existing unassigned node for 70236052 that is in expected state RS_ZK_REGION_OPENED 2012-01-19 11:12:37,249 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:20000-0x334cb6fb0f0011e Successfully deleted unassigned node for region 70236052 in expected state RS_ZK_REGION_OPENED 2012-01-19 11:12:37,250 INFO org.apache.hadoop.hbase.master.handler.OpenedRegionHandler: The master has opened the region -ROOT-,,0.70236052 that was online on serverName=C3S32,20020,1326987400439, load=(requests=0, regions=0, usedHeap=32, maxHeap=991) 2012-01-19 11:12:37,250 INFO org.apache.hadoop.hbase.master.HMaster: -ROOT- assigned=1, rit=false, location=C3S32:20020 2012-01-19 11:12:37,280 INFO org.apache.hadoop.ipc.HbaseRPC: Server at C3S33/158.1.130.33:20020 could not be reached after 1 tries, giving up. 2012-01-19 11:12:37,281 INFO org.apache.hadoop.hbase.catalog.CatalogTracker: Passed metaserver is null 2012-01-19 11:12:37,283 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:20000-0x334cb6fb0f0011e Creating (or updating) unassigned node for 1028785192 with OFFLINE state 2012-01-19 11:12:37,294 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=M_ZK_REGION_OFFLINE, server=C3S31:20000, region=1028785192/.META. 2012-01-19 11:12:37,294 INFO org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan was found (or we are ignoring an existing plan) for .META.,,1.1028785192 so generated a random one; hri=.META.,,1.1028785192, src=, dest=C3S32,20020,1326987400439; 1 (online=1, exclude=null) available servers 2012-01-19 11:12:37,294 INFO org.apache.hadoop.hbase.master.AssignmentManager: Assigning region .META.,,1.1028785192 to C3S32,20020,1326987400439 2012-01-19 11:12:37,307 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_OPENING, server=C3S32,20020,1326987400439, region=1028785192/.META. 2012-01-19 11:12:37,335 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_OPENING, server=C3S32,20020,1326987400439, region=1028785192/.META. 2012-01-19 11:12:37,361 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling transition=RS_ZK_REGION_OPENED, server=C3S32,20020,1326987400439, region=1028785192/.META. 2012-01-19 11:12:37,362 INFO org.apache.hadoop.hbase.master.handler.OpenedRegionHandler: Handling OPENED event for .META.,,1.1028785192 from serverName=C3S32,20020,1326987400439, load=(requests=0, regions=0, usedHeap=32, maxHeap=991); deleting unassigned node 2012-01-19 11:12:37,363 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:20000-0x334cb6fb0f0011e Deleting existing unassigned node for 1028785192 that is in expected state RS_ZK_REGION_OPENED 2012-01-19 11:12:37,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:20000-0x334cb6fb0f0011e Successfully deleted unassigned node for region 1028785192 in expected state RS_ZK_REGION_OPENED 2012-01-19 11:12:37,372 INFO org.apache.hadoop.hbase.master.handler.OpenedRegionHandler: The master has opened the region .META.,,1.1028785192 that was online on serverName=C3S32,20020,1326987400439, load=(requests=0, regions=0, usedHeap=32, maxHeap=991) 2012-01-19 11:12:37,373 INFO org.apache.hadoop.hbase.zookeeper.MetaNodeTracker: Detected completed assignment of META, notifying catalog tracker 2012-01-19 11:12:37,382 INFO org.apache.hadoop.hbase.master.HMaster: .META. assigned=2, rit=false, location=C3S32:20020 2012-01-19 11:12:37,382 INFO org.apache.hadoop.hbase.master.HMaster: Master startup proceeding: master failover 2012-01-19 11:12:37,383 WARN org.apache.hadoop.hbase.master.AssignmentManager: Overwriting 1028785192 on serverName=C3S32,20020,1326987400439, load=(requests=0, regions=0, usedHeap=0, maxHeap=0) 2012-01-19 11:12:37,384 WARN org.apache.hadoop.hbase.master.AssignmentManager: Overwriting 70236052 on serverName=C3S32,20020,1326987400439, load=(requests=0, regions=0, usedHeap=0, maxHeap=0) 2012-01-19 11:12:37,393 INFO org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Reassigning 0 region(s) that C3S31,20020,1326987900492 was carrying (skipping 0 regions(s) that are already in transition) 2012-01-19 11:12:37,393 INFO org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Finished processing of shutdown of C3S31,20020,1326987900492 2012-01-19 11:12:37,398 INFO org.apache.hadoop.hbase.master.AssignmentManager: No regions in transition in ZK to process on failover 2012-01-19 11:12:37,408 INFO org.apache.hadoop.hbase.master.HMaster: Master has completed initialization 2012-01-19 11:12:37,417 DEBUG org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 0 catalog row(s) and gc'd 0 unreferenced parent region(s) 2012-01-19 11:12:49,855 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Splitting 1 hlog(s) in hdfs://C3S31:9000/hbase/.logs/C3S33,20020,1326987399579 2012-01-19 11:12:49,857 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Writer thread Thread[WriterThread-0,5,main]: starting 2012-01-19 11:12:49,857 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Writer thread Thread[WriterThread-1,5,main]: starting 2012-01-19 11:12:49,859 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Writer thread Thread[WriterThread-2,5,main]: starting 2012-01-19 11:12:49,860 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Writer thread Thread[WriterThread-4,5,main]: starting 2012-01-19 11:12:49,860 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Writer thread Thread[WriterThread-5,5,main]: starting 2012-01-19 11:12:49,860 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Splitting hlog 1 of 1: hdfs://C3S31:9000/hbase/.logs/C3S33,20020,1326987399579/C3S33%3A20020.1326987400363, length=0 2012-01-19 11:12:49,860 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Writer thread Thread[WriterThread-3,5,main]: starting 2012-01-19 11:12:49,861 INFO org.apache.hadoop.hbase.util.FSUtils: Recovering file hdfs://C3S31:9000/hbase/.logs/C3S33,20020,1326987399579/C3S33%3A20020.1326987400363 2012-01-19 11:12:50,863 INFO org.apache.hadoop.hbase.util.FSUtils: Finished lease recover attempt for hdfs://C3S31:9000/hbase/.logs/C3S33,20020,1326987399579/C3S33%3A20020.1326987400363 2012-01-19 11:12:50,863 WARN org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: File hdfs://C3S31:9000/hbase/.logs/C3S33,20020,1326987399579/C3S33%3A20020.1326987400363 might be still open, length is 0 2012-01-19 11:12:50,880 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Pushed=0 entries from hdfs://C3S31:9000/hbase/.logs/C3S33,20020,1326987399579/C3S33%3A20020.1326987400363 2012-01-19 11:12:50,882 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Waiting for split writer threads to finish 2012-01-19 11:12:50,884 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Split writers finished 2012-01-19 11:12:50,893 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived processed log hdfs://C3S31:9000/hbase/.logs/C3S33,20020,1326987399579/C3S33%3A20020.1326987400363 to hdfs://C3S31:9000/hbase/.oldlogs/C3S33%3A20020.1326987400363 2012-01-19 11:12:50,895 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: hlog file splitting completed in 1044 ms for hdfs://C3S31:9000/hbase/.logs/C3S33,20020,1326987399579 2012-01-19 11:12:50,907 INFO org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Reassigning 0 region(s) that C3S33,20020,1326987399579 was carrying (skipping 0 regions(s) that are already in transition) 2012-01-19 11:12:50,907 INFO org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Finished processing of shutdown of C3S33,20020,1326987399579 2012-01-19 11:15:20,770 DEBUG org.apache.hadoop.hbase.client.MetaScanner: Scanning .META. starting at row= for max=2147483647 rows 2012-01-19 11:15:20,772 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@4e4b12d9; hsa=C3S32:20020 2012-01-19 11:15:20,784 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Cached location for .META.,,1.1028785192 is C3S32:20020 2012-01-19 11:15:20,791 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: The connection to hconnection-0x334cb6fb0f0011f has been closed. 2012-01-19 11:17:37,411 DEBUG org.apache.hadoop.hbase.master.LoadBalancer: Server information: C3S32,20020,1326987400439=2 2012-01-19 11:17:37,411 INFO org.apache.hadoop.hbase.master.LoadBalancer: Skipping load balancing. servers=1 regions=2 average=2.0 mostloaded=2 leastloaded=2 2012-01-19 11:17:37,423 DEBUG org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 0 catalog row(s) and gc'd 0 unreferenced parent region(s) 2012-01-19 11:22:37,409 DEBUG org.apache.hadoop.hbase.master.LoadBalancer: Server information: C3S32,20020,1326987400439=2 2012-01-19 11:22:37,409 INFO org.apache.hadoop.hbase.master.LoadBalancer: Skipping load balancing. servers=1 regions=2 average=2.0 mostloaded=2 leastloaded=2 2012-01-19 11:22:37,421 DEBUG org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 0 catalog row(s) and gc'd 0 unreferenced parent region(s)