2016-01-11 19:20:56,277 INFO [main] hbase.ResourceChecker(147): before: regionserver.TestSplitWalDataLoss#test Thread=4, OpenFileDescriptor=168, MaxFileDescriptor=10240, SystemLoadAverage=985, ProcessCount=288, AvailableMemoryMB=133, ConnectionCount=0 2016-01-11 19:20:56,283 INFO [main] hbase.HBaseTestingUtility(967): Starting up minicluster with 1 master(s) and 2 regionserver(s) and 2 datanode(s) 2016-01-11 19:20:56,311 INFO [main] hbase.HBaseTestingUtility(447): Created new mini-cluster data directory: /Users/elliott/Code/Public/hbase/hbase-server/target/test-data/e0ff3de9-2b27-45bb-ae9b-800547bf5540/dfscluster_d09e3e7b-f697-4440-a7e3-d570985bf1a3, deleteOnExit=true 2016-01-11 19:20:56,312 INFO [main] hbase.HBaseTestingUtility(697): Setting test.cache.data to /Users/elliott/Code/Public/hbase/hbase-server/target/test-data/e0ff3de9-2b27-45bb-ae9b-800547bf5540/cache_data in system properties and HBase conf 2016-01-11 19:20:56,313 INFO [main] hbase.HBaseTestingUtility(697): Setting hadoop.tmp.dir to /Users/elliott/Code/Public/hbase/hbase-server/target/test-data/e0ff3de9-2b27-45bb-ae9b-800547bf5540/hadoop_tmp in system properties and HBase conf 2016-01-11 19:20:56,313 INFO [main] hbase.HBaseTestingUtility(697): Setting hadoop.log.dir to /Users/elliott/Code/Public/hbase/hbase-server/target/test-data/e0ff3de9-2b27-45bb-ae9b-800547bf5540/hadoop_logs in system properties and HBase conf 2016-01-11 19:20:56,314 INFO [main] hbase.HBaseTestingUtility(697): Setting mapreduce.cluster.local.dir to /Users/elliott/Code/Public/hbase/hbase-server/target/test-data/e0ff3de9-2b27-45bb-ae9b-800547bf5540/mapred_local in system properties and HBase conf 2016-01-11 19:20:56,314 INFO [main] hbase.HBaseTestingUtility(697): Setting mapreduce.cluster.temp.dir to /Users/elliott/Code/Public/hbase/hbase-server/target/test-data/e0ff3de9-2b27-45bb-ae9b-800547bf5540/mapred_temp in system properties and HBase conf 2016-01-11 19:20:56,314 INFO [main] hbase.HBaseTestingUtility(688): read short circuit is OFF 2016-01-11 19:20:56,442 WARN [main] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2016-01-11 19:20:56,584 DEBUG [main] fs.HFileSystem(221): The file system is not a DistributedFileSystem. Skipping on block location reordering Formatting using clusterid: testClusterID 2016-01-11 19:20:57,255 WARN [main] impl.MetricsConfig(124): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2016-01-11 19:20:57,392 INFO [main] log.Slf4jLog(67): Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2016-01-11 19:20:57,445 INFO [main] log.Slf4jLog(67): jetty-6.1.26 2016-01-11 19:20:57,502 INFO [main] log.Slf4jLog(67): Extract jar:file:/Users/elliott/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.5.1/hadoop-hdfs-2.5.1-tests.jar!/webapps/hdfs to /var/folders/tn/pyvx59yd0dz6td198ys7ffhj_pvvgm/T/Jetty_localhost_58682_hdfs____5s2ac7/webapp 2016-01-11 19:20:57,644 WARN [main] server.AuthenticationFilter(166): 'signature.secret' configuration not set, using a random value as secret 2016-01-11 19:20:57,677 INFO [main] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58682 2016-01-11 19:20:58,177 INFO [main] log.Slf4jLog(67): jetty-6.1.26 2016-01-11 19:20:58,179 INFO [main] log.Slf4jLog(67): Extract jar:file:/Users/elliott/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.5.1/hadoop-hdfs-2.5.1-tests.jar!/webapps/datanode to /var/folders/tn/pyvx59yd0dz6td198ys7ffhj_pvvgm/T/Jetty_localhost_58685_datanode____qf13iv/webapp 2016-01-11 19:20:58,250 INFO [main] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58685 2016-01-11 19:20:58,308 INFO [main] log.Slf4jLog(67): jetty-6.1.26 2016-01-11 19:20:58,312 INFO [main] log.Slf4jLog(67): Extract jar:file:/Users/elliott/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.5.1/hadoop-hdfs-2.5.1-tests.jar!/webapps/datanode to /var/folders/tn/pyvx59yd0dz6td198ys7ffhj_pvvgm/T/Jetty_localhost_58688_datanode____buz5wa/webapp 2016-01-11 19:20:58,394 INFO [main] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58688 2016-01-11 19:20:58,642 INFO [IPC Server handler 6 on 58683] blockmanagement.BlockManager(1753): BLOCK* processReport: from storage DS-84b79f3b-7553-476e-b313-9382dc0bd9af node DatanodeRegistration(127.0.0.1, datanodeUuid=1c4e8d3e-68ce-4377-bddf-c64ec7768a10, infoPort=58685, ipcPort=58686, storageInfo=lv=-55;cid=testClusterID;nsid=722960465;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs 2016-01-11 19:20:58,642 INFO [IPC Server handler 7 on 58683] blockmanagement.BlockManager(1753): BLOCK* processReport: from storage DS-70606f4d-e6bd-4292-9765-171d84a1305c node DatanodeRegistration(127.0.0.1, datanodeUuid=39ef380c-c889-4f94-ac05-c6b51ea2ad87, infoPort=58688, ipcPort=58690, storageInfo=lv=-55;cid=testClusterID;nsid=722960465;c=0), blocks: 0, hasStaleStorages: true, processing time: 1 msecs 2016-01-11 19:20:58,642 INFO [IPC Server handler 6 on 58683] blockmanagement.BlockManager(1753): BLOCK* processReport: from storage DS-6ab76dd9-6c4e-41f4-9b47-7b827e25bec7 node DatanodeRegistration(127.0.0.1, datanodeUuid=1c4e8d3e-68ce-4377-bddf-c64ec7768a10, infoPort=58685, ipcPort=58686, storageInfo=lv=-55;cid=testClusterID;nsid=722960465;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs 2016-01-11 19:20:58,642 INFO [IPC Server handler 7 on 58683] blockmanagement.BlockManager(1753): BLOCK* processReport: from storage DS-222d7346-39ee-452c-9ca0-e0cb04e5fd92 node DatanodeRegistration(127.0.0.1, datanodeUuid=39ef380c-c889-4f94-ac05-c6b51ea2ad87, infoPort=58688, ipcPort=58690, storageInfo=lv=-55;cid=testClusterID;nsid=722960465;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs 2016-01-11 19:20:58,791 INFO [main] zookeeper.MiniZooKeeperCluster(273): Started MiniZooKeeperCluster and ran successful 'stat' on client port=54890 2016-01-11 19:20:58,804 INFO [main] fs.HFileSystem(252): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2016-01-11 19:20:58,806 INFO [main] fs.HFileSystem(252): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2016-01-11 19:20:59,011 INFO [IPC Server handler 7 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58687 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6ab76dd9-6c4e-41f4-9b47-7b827e25bec7:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-70606f4d-e6bd-4292-9765-171d84a1305c:NORMAL|RBW]]} size 0 2016-01-11 19:20:59,016 INFO [IPC Server handler 6 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58684 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6ab76dd9-6c4e-41f4-9b47-7b827e25bec7:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-70606f4d-e6bd-4292-9765-171d84a1305c:NORMAL|RBW]]} size 0 2016-01-11 19:20:59,026 INFO [main] util.FSUtils(747): Created version file at hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb with version=8 2016-01-11 19:20:59,382 INFO [main] client.ConnectionUtils(104): master/localhost/127.0.0.1:0 server-side HConnection retries=350 2016-01-11 19:20:59,693 INFO [main] ipc.SimpleRpcScheduler(128): Using deadline as user call queue, count=1 2016-01-11 19:20:59,702 INFO [main] ipc.RpcServer$Listener(637): master/localhost/127.0.0.1:0: started 3 reader(s) listening on port=58695 2016-01-11 19:20:59,741 INFO [main] fs.HFileSystem(252): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2016-01-11 19:20:59,800 INFO [main] zookeeper.RecoverableZooKeeper(120): Process identifier=master:58695 connecting to ZooKeeper ensemble=localhost:54890 2016-01-11 19:20:59,833 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:586950x0, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2016-01-11 19:20:59,835 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(638): master:58695-0x15233d9d0910000 connected 2016-01-11 19:20:59,872 DEBUG [main] zookeeper.ZKUtil(370): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2016-01-11 19:20:59,873 DEBUG [main] zookeeper.ZKUtil(370): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2016-01-11 19:20:59,876 INFO [RpcServer.responder] ipc.RpcServer$Responder(956): RpcServer.responder: starting 2016-01-11 19:20:59,876 INFO [RpcServer.listener,port=58695] ipc.RpcServer$Listener(768): RpcServer.listener,port=58695: starting 2016-01-11 19:20:59,877 DEBUG [main] ipc.RpcExecutor(115): B.default Start Handler index=0 queue=0 2016-01-11 19:20:59,877 DEBUG [main] ipc.RpcExecutor(115): B.default Start Handler index=1 queue=0 2016-01-11 19:20:59,877 DEBUG [main] ipc.RpcExecutor(115): B.default Start Handler index=2 queue=0 2016-01-11 19:20:59,878 DEBUG [main] ipc.RpcExecutor(115): B.default Start Handler index=3 queue=0 2016-01-11 19:20:59,878 DEBUG [main] ipc.RpcExecutor(115): B.default Start Handler index=4 queue=0 2016-01-11 19:20:59,878 DEBUG [main] ipc.RpcExecutor(115): Priority Start Handler index=0 queue=0 2016-01-11 19:20:59,879 DEBUG [main] ipc.RpcExecutor(115): Priority Start Handler index=1 queue=1 2016-01-11 19:20:59,879 DEBUG [main] ipc.RpcExecutor(115): Priority Start Handler index=2 queue=0 2016-01-11 19:20:59,879 DEBUG [main] ipc.RpcExecutor(115): Priority Start Handler index=3 queue=1 2016-01-11 19:20:59,880 DEBUG [main] ipc.RpcExecutor(115): Priority Start Handler index=4 queue=0 2016-01-11 19:20:59,880 DEBUG [main] ipc.RpcExecutor(115): Replication Start Handler index=0 queue=0 2016-01-11 19:20:59,880 DEBUG [main] ipc.RpcExecutor(115): Replication Start Handler index=1 queue=0 2016-01-11 19:20:59,881 DEBUG [main] ipc.RpcExecutor(115): Replication Start Handler index=2 queue=0 2016-01-11 19:20:59,890 INFO [main] master.HMaster(367): hbase.rootdir=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb, hbase.cluster.distributed=false 2016-01-11 19:20:59,907 INFO [main] master.HMaster(1705): Adding backup master ZNode /hbase/backup-masters/localhost,58695,1452568859727 2016-01-11 19:20:59,919 DEBUG [main] zookeeper.ZKUtil(368): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/localhost,58695,1452568859727 2016-01-11 19:20:59,925 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2016-01-11 19:20:59,926 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(368): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2016-01-11 19:20:59,927 INFO [localhost:58695.activeMasterManager] master.ActiveMasterManager(170): Deleting ZNode for /hbase/backup-masters/localhost,58695,1452568859727 from backup master directory 2016-01-11 19:20:59,929 DEBUG [main-EventThread] zookeeper.ZKUtil(368): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2016-01-11 19:20:59,929 DEBUG [main-EventThread] master.ActiveMasterManager(126): A master is now available 2016-01-11 19:20:59,929 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/localhost,58695,1452568859727 2016-01-11 19:20:59,931 WARN [localhost:58695.activeMasterManager] hbase.ZNodeClearer(58): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2016-01-11 19:20:59,931 INFO [localhost:58695.activeMasterManager] master.ActiveMasterManager(179): Registered Active Master=localhost,58695,1452568859727 2016-01-11 19:20:59,980 INFO [main] client.ConnectionUtils(104): regionserver/localhost/127.0.0.1:0 server-side HConnection retries=350 2016-01-11 19:20:59,980 INFO [main] ipc.SimpleRpcScheduler(128): Using deadline as user call queue, count=1 2016-01-11 19:20:59,981 INFO [main] ipc.RpcServer$Listener(637): regionserver/localhost/127.0.0.1:0: started 3 reader(s) listening on port=58697 2016-01-11 19:20:59,984 INFO [main] fs.HFileSystem(252): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2016-01-11 19:20:59,986 INFO [main] zookeeper.RecoverableZooKeeper(120): Process identifier=regionserver:58697 connecting to ZooKeeper ensemble=localhost:54890 2016-01-11 19:20:59,990 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): regionserver:586970x0, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2016-01-11 19:20:59,991 DEBUG [main] zookeeper.ZKUtil(368): regionserver:586970x0, quorum=localhost:54890, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2016-01-11 19:20:59,992 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(638): regionserver:58697-0x15233d9d0910001 connected 2016-01-11 19:20:59,992 DEBUG [main] zookeeper.ZKUtil(370): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2016-01-11 19:20:59,993 INFO [RpcServer.responder] ipc.RpcServer$Responder(956): RpcServer.responder: starting 2016-01-11 19:20:59,993 INFO [RpcServer.listener,port=58697] ipc.RpcServer$Listener(768): RpcServer.listener,port=58697: starting 2016-01-11 19:20:59,993 DEBUG [main] ipc.RpcExecutor(115): B.default Start Handler index=0 queue=0 2016-01-11 19:20:59,994 DEBUG [main] ipc.RpcExecutor(115): B.default Start Handler index=1 queue=0 2016-01-11 19:20:59,994 DEBUG [main] ipc.RpcExecutor(115): B.default Start Handler index=2 queue=0 2016-01-11 19:20:59,994 DEBUG [main] ipc.RpcExecutor(115): B.default Start Handler index=3 queue=0 2016-01-11 19:20:59,994 DEBUG [main] ipc.RpcExecutor(115): B.default Start Handler index=4 queue=0 2016-01-11 19:20:59,995 DEBUG [main] ipc.RpcExecutor(115): Priority Start Handler index=0 queue=0 2016-01-11 19:20:59,996 DEBUG [main] ipc.RpcExecutor(115): Priority Start Handler index=1 queue=1 2016-01-11 19:20:59,996 DEBUG [main] ipc.RpcExecutor(115): Priority Start Handler index=2 queue=0 2016-01-11 19:20:59,996 DEBUG [main] ipc.RpcExecutor(115): Priority Start Handler index=3 queue=1 2016-01-11 19:20:59,997 DEBUG [main] ipc.RpcExecutor(115): Priority Start Handler index=4 queue=0 2016-01-11 19:20:59,997 DEBUG [main] ipc.RpcExecutor(115): Replication Start Handler index=0 queue=0 2016-01-11 19:20:59,997 DEBUG [main] ipc.RpcExecutor(115): Replication Start Handler index=1 queue=0 2016-01-11 19:20:59,997 DEBUG [main] ipc.RpcExecutor(115): Replication Start Handler index=2 queue=0 2016-01-11 19:21:00,023 INFO [main] client.ConnectionUtils(104): regionserver/localhost/127.0.0.1:0 server-side HConnection retries=350 2016-01-11 19:21:00,024 INFO [main] ipc.SimpleRpcScheduler(128): Using deadline as user call queue, count=1 2016-01-11 19:21:00,026 INFO [main] ipc.RpcServer$Listener(637): regionserver/localhost/127.0.0.1:0: started 3 reader(s) listening on port=58702 2016-01-11 19:21:00,026 INFO [IPC Server handler 5 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58687 is added to blk_1073741826_1002{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-222d7346-39ee-452c-9ca0-e0cb04e5fd92:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-84b79f3b-7553-476e-b313-9382dc0bd9af:NORMAL|RBW]]} size 0 2016-01-11 19:21:00,028 INFO [IPC Server handler 7 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58684 is added to blk_1073741826_1002{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-222d7346-39ee-452c-9ca0-e0cb04e5fd92:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-84b79f3b-7553-476e-b313-9382dc0bd9af:NORMAL|RBW]]} size 0 2016-01-11 19:21:00,029 INFO [main] fs.HFileSystem(252): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2016-01-11 19:21:00,030 DEBUG [localhost:58695.activeMasterManager] util.FSUtils(899): Created cluster ID file at hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/hbase.id with ID: 2997bd1b-32c1-4416-b00d-6f56c9f9192b 2016-01-11 19:21:00,032 INFO [main] zookeeper.RecoverableZooKeeper(120): Process identifier=regionserver:58702 connecting to ZooKeeper ensemble=localhost:54890 2016-01-11 19:21:00,036 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): regionserver:587020x0, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2016-01-11 19:21:00,036 DEBUG [main] zookeeper.ZKUtil(368): regionserver:587020x0, quorum=localhost:54890, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2016-01-11 19:21:00,038 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(638): regionserver:58702-0x15233d9d0910002 connected 2016-01-11 19:21:00,039 DEBUG [main] zookeeper.ZKUtil(370): regionserver:58702-0x15233d9d0910002, quorum=localhost:54890, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2016-01-11 19:21:00,039 INFO [RpcServer.responder] ipc.RpcServer$Responder(956): RpcServer.responder: starting 2016-01-11 19:21:00,039 INFO [RpcServer.listener,port=58702] ipc.RpcServer$Listener(768): RpcServer.listener,port=58702: starting 2016-01-11 19:21:00,039 DEBUG [main] ipc.RpcExecutor(115): B.default Start Handler index=0 queue=0 2016-01-11 19:21:00,040 DEBUG [main] ipc.RpcExecutor(115): B.default Start Handler index=1 queue=0 2016-01-11 19:21:00,041 DEBUG [main] ipc.RpcExecutor(115): B.default Start Handler index=2 queue=0 2016-01-11 19:21:00,041 DEBUG [main] ipc.RpcExecutor(115): B.default Start Handler index=3 queue=0 2016-01-11 19:21:00,041 DEBUG [main] ipc.RpcExecutor(115): B.default Start Handler index=4 queue=0 2016-01-11 19:21:00,042 DEBUG [main] ipc.RpcExecutor(115): Priority Start Handler index=0 queue=0 2016-01-11 19:21:00,042 DEBUG [main] ipc.RpcExecutor(115): Priority Start Handler index=1 queue=1 2016-01-11 19:21:00,042 DEBUG [main] ipc.RpcExecutor(115): Priority Start Handler index=2 queue=0 2016-01-11 19:21:00,043 DEBUG [main] ipc.RpcExecutor(115): Priority Start Handler index=3 queue=1 2016-01-11 19:21:00,043 DEBUG [main] ipc.RpcExecutor(115): Priority Start Handler index=4 queue=0 2016-01-11 19:21:00,043 DEBUG [main] ipc.RpcExecutor(115): Replication Start Handler index=0 queue=0 2016-01-11 19:21:00,043 DEBUG [main] ipc.RpcExecutor(115): Replication Start Handler index=1 queue=0 2016-01-11 19:21:00,044 DEBUG [main] ipc.RpcExecutor(115): Replication Start Handler index=2 queue=0 2016-01-11 19:21:00,079 INFO [localhost:58695.activeMasterManager] master.MasterFileSystem(531): BOOTSTRAP: creating hbase:meta region 2016-01-11 19:21:00,081 INFO [localhost:58695.activeMasterManager] regionserver.HRegion(6219): creating HRegion hbase:meta HTD == 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}, {NAME => 'info', BLOOMFILTER => 'NONE', VERSIONS => '10', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'false', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} RootDir = hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb Table name == hbase:meta 2016-01-11 19:21:00,085 INFO [RS:1;localhost:58702] zookeeper.RecoverableZooKeeper(120): Process identifier=hconnection-0x4ec975e2 connecting to ZooKeeper ensemble=localhost:54890 2016-01-11 19:21:00,085 INFO [M:0;localhost:58695] zookeeper.RecoverableZooKeeper(120): Process identifier=hconnection-0x5fbed0e5 connecting to ZooKeeper ensemble=localhost:54890 2016-01-11 19:21:00,085 INFO [RS:0;localhost:58697] zookeeper.RecoverableZooKeeper(120): Process identifier=hconnection-0x7d0f39b4 connecting to ZooKeeper ensemble=localhost:54890 2016-01-11 19:21:00,090 DEBUG [RS:0;localhost:58697-EventThread] zookeeper.ZooKeeperWatcher(554): hconnection-0x7d0f39b40x0, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2016-01-11 19:21:00,091 DEBUG [RS:1;localhost:58702-EventThread] zookeeper.ZooKeeperWatcher(554): hconnection-0x4ec975e20x0, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2016-01-11 19:21:00,092 DEBUG [M:0;localhost:58695-EventThread] zookeeper.ZooKeeperWatcher(554): hconnection-0x5fbed0e50x0, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2016-01-11 19:21:00,092 INFO [RS:0;localhost:58697] client.ZooKeeperRegistry(107): ClusterId read in ZooKeeper is null 2016-01-11 19:21:00,092 INFO [M:0;localhost:58695] client.ZooKeeperRegistry(107): ClusterId read in ZooKeeper is null 2016-01-11 19:21:00,092 DEBUG [RS:0;localhost:58697-EventThread] zookeeper.ZooKeeperWatcher(638): hconnection-0x7d0f39b4-0x15233d9d0910003 connected 2016-01-11 19:21:00,092 INFO [RS:1;localhost:58702] client.ZooKeeperRegistry(107): ClusterId read in ZooKeeper is null 2016-01-11 19:21:00,093 DEBUG [RS:1;localhost:58702] client.ConnectionManager$HConnectionImplementation(911): clusterid came back null, using default default-cluster 2016-01-11 19:21:00,093 DEBUG [M:0;localhost:58695-EventThread] zookeeper.ZooKeeperWatcher(638): hconnection-0x5fbed0e5-0x15233d9d0910005 connected 2016-01-11 19:21:00,093 DEBUG [RS:1;localhost:58702-EventThread] zookeeper.ZooKeeperWatcher(638): hconnection-0x4ec975e2-0x15233d9d0910004 connected 2016-01-11 19:21:00,092 DEBUG [M:0;localhost:58695] client.ConnectionManager$HConnectionImplementation(911): clusterid came back null, using default default-cluster 2016-01-11 19:21:00,092 DEBUG [RS:0;localhost:58697] client.ConnectionManager$HConnectionImplementation(911): clusterid came back null, using default default-cluster 2016-01-11 19:21:00,103 DEBUG [M:0;localhost:58695] ipc.AbstractRpcClient(115): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3954fc4c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2016-01-11 19:21:00,103 DEBUG [RS:0;localhost:58697] ipc.AbstractRpcClient(115): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@73c8d865, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2016-01-11 19:21:00,103 DEBUG [RS:1;localhost:58702] ipc.AbstractRpcClient(115): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@400e824b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2016-01-11 19:21:00,117 INFO [IPC Server handler 9 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58687 is added to blk_1073741827_1003{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6ab76dd9-6c4e-41f4-9b47-7b827e25bec7:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-70606f4d-e6bd-4292-9765-171d84a1305c:NORMAL|RBW]]} size 0 2016-01-11 19:21:00,117 INFO [IPC Server handler 1 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58684 is added to blk_1073741827_1003{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6ab76dd9-6c4e-41f4-9b47-7b827e25bec7:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-70606f4d-e6bd-4292-9765-171d84a1305c:NORMAL|RBW]]} size 0 2016-01-11 19:21:00,121 DEBUG [localhost:58695.activeMasterManager] regionserver.HRegion(733): Instantiated hbase:meta,,1.1588230740 2016-01-11 19:21:00,161 INFO [StoreOpener-1588230740-1] hfile.CacheConfig(548): Allocating LruBlockCache size=995.60 MB, blockSize=64 KB 2016-01-11 19:21:00,169 DEBUG [StoreOpener-1588230740-1] hfile.CacheConfig(562): Trying to use Internal l2 cache 2016-01-11 19:21:00,169 INFO [StoreOpener-1588230740-1] hfile.CacheConfig(292): blockCache=LruBlockCache{blockCount=0, currentSize=1071840, freeSize=1042890464, maxSize=1043962304, heapSize=1071840, minSize=991764160, minFactor=0.95, multiSize=495882080, multiFactor=0.5, singleSize=247941040, singleFactor=0.25}, cacheDataOnRead=false, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 2016-01-11 19:21:00,179 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(104): size [134217728, 9223372036854775807, 9223372036854775807); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000 2016-01-11 19:21:00,181 DEBUG [StoreOpener-1588230740-1] regionserver.HRegionFileSystem(201): No StoreFiles for: hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/meta/1588230740/info 2016-01-11 19:21:00,196 DEBUG [localhost:58695.activeMasterManager] regionserver.HRegion(3890): Found 0 recovered edits file(s) under hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/meta/1588230740 2016-01-11 19:21:00,205 DEBUG [localhost:58695.activeMasterManager] regionserver.FlushLargeStoresPolicy(56): hbase.hregion.percolumnfamilyflush.size.lower.bound is not specified, use global config(16777216) instead 2016-01-11 19:21:00,211 DEBUG [localhost:58695.activeMasterManager] wal.WALSplitter(729): Wrote region seqId=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/meta/1588230740/recovered.edits/2.seqid to file, newSeqId=2, maxSeqId=0 2016-01-11 19:21:00,211 INFO [localhost:58695.activeMasterManager] regionserver.HRegion(868): Onlined 1588230740; next sequenceid=2 2016-01-11 19:21:00,212 DEBUG [localhost:58695.activeMasterManager] regionserver.HRegion(1389): Closing hbase:meta,,1.1588230740: disabling compactions & flushes 2016-01-11 19:21:00,212 DEBUG [localhost:58695.activeMasterManager] regionserver.HRegion(1416): Updates disabled for region hbase:meta,,1.1588230740 2016-01-11 19:21:00,213 INFO [StoreCloserThread-hbase:meta,,1.1588230740-1] regionserver.HStore(875): Closed info 2016-01-11 19:21:00,213 INFO [localhost:58695.activeMasterManager] regionserver.HRegion(1519): Closed hbase:meta,,1.1588230740 2016-01-11 19:21:00,243 INFO [IPC Server handler 1 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58687 is added to blk_1073741828_1004{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-84b79f3b-7553-476e-b313-9382dc0bd9af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-222d7346-39ee-452c-9ca0-e0cb04e5fd92:NORMAL|RBW]]} size 0 2016-01-11 19:21:00,244 INFO [IPC Server handler 0 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58684 is added to blk_1073741828_1004{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-84b79f3b-7553-476e-b313-9382dc0bd9af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-222d7346-39ee-452c-9ca0-e0cb04e5fd92:NORMAL|RBW]]} size 0 2016-01-11 19:21:00,247 DEBUG [localhost:58695.activeMasterManager] util.FSTableDescriptors(656): Wrote descriptor into: hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2016-01-11 19:21:00,258 INFO [localhost:58695.activeMasterManager] fs.HFileSystem(252): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2016-01-11 19:21:00,264 INFO [localhost:58695.activeMasterManager] coordination.ZKSplitLogManagerCoordination(599): Found 0 orphan tasks and 0 rescan nodes 2016-01-11 19:21:00,265 DEBUG [localhost:58695.activeMasterManager] util.FSTableDescriptors(204): Fetching table descriptors from the filesystem. 2016-01-11 19:21:00,279 INFO [localhost:58695.activeMasterManager] zookeeper.RecoverableZooKeeper(120): Process identifier=hconnection-0x67a32ec9 connecting to ZooKeeper ensemble=localhost:54890 2016-01-11 19:21:00,282 DEBUG [localhost:58695.activeMasterManager-EventThread] zookeeper.ZooKeeperWatcher(554): hconnection-0x67a32ec90x0, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2016-01-11 19:21:00,283 DEBUG [localhost:58695.activeMasterManager] ipc.AbstractRpcClient(115): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@23e2d70d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2016-01-11 19:21:00,284 DEBUG [localhost:58695.activeMasterManager-EventThread] zookeeper.ZooKeeperWatcher(638): hconnection-0x67a32ec9-0x15233d9d0910006 connected 2016-01-11 19:21:00,292 INFO [localhost:58695.activeMasterManager] balancer.StochasticLoadBalancer(135): loading config 2016-01-11 19:21:00,302 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(370): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2016-01-11 19:21:00,303 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(370): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2016-01-11 19:21:00,331 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2016-01-11 19:21:00,331 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2016-01-11 19:21:00,331 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): regionserver:58702-0x15233d9d0910002, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2016-01-11 19:21:00,331 INFO [localhost:58695.activeMasterManager] master.HMaster(586): Server active/primary master=localhost,58695,1452568859727, sessionid=0x15233d9d0910000, setting cluster-up flag (Was=false) 2016-01-11 19:21:00,332 INFO [M:0;localhost:58695] regionserver.HRegionServer(782): ClusterId : 2997bd1b-32c1-4416-b00d-6f56c9f9192b 2016-01-11 19:21:00,332 INFO [RS:0;localhost:58697] regionserver.HRegionServer(782): ClusterId : 2997bd1b-32c1-4416-b00d-6f56c9f9192b 2016-01-11 19:21:00,333 INFO [RS:1;localhost:58702] regionserver.HRegionServer(782): ClusterId : 2997bd1b-32c1-4416-b00d-6f56c9f9192b 2016-01-11 19:21:00,339 DEBUG [RS:0;localhost:58697] procedure.RegionServerProcedureManagerHost(44): Procedure flush-table-proc is initializing 2016-01-11 19:21:00,339 DEBUG [RS:1;localhost:58702] procedure.RegionServerProcedureManagerHost(44): Procedure flush-table-proc is initializing 2016-01-11 19:21:00,345 DEBUG [RS:0;localhost:58697] zookeeper.RecoverableZooKeeper(594): Node /hbase/flush-table-proc already exists 2016-01-11 19:21:00,346 DEBUG [RS:0;localhost:58697] zookeeper.RecoverableZooKeeper(594): Node /hbase/flush-table-proc/acquired already exists 2016-01-11 19:21:00,348 DEBUG [RS:0;localhost:58697] zookeeper.RecoverableZooKeeper(594): Node /hbase/flush-table-proc/abort already exists 2016-01-11 19:21:00,349 DEBUG [localhost:58695.activeMasterManager] zookeeper.RecoverableZooKeeper(594): Node /hbase/flush-table-proc/acquired already exists 2016-01-11 19:21:00,349 DEBUG [RS:1;localhost:58702] procedure.RegionServerProcedureManagerHost(46): Procedure flush-table-proc is initialized 2016-01-11 19:21:00,349 DEBUG [RS:0;localhost:58697] procedure.RegionServerProcedureManagerHost(46): Procedure flush-table-proc is initialized 2016-01-11 19:21:00,350 DEBUG [RS:0;localhost:58697] procedure.RegionServerProcedureManagerHost(44): Procedure online-snapshot is initializing 2016-01-11 19:21:00,349 DEBUG [RS:1;localhost:58702] procedure.RegionServerProcedureManagerHost(44): Procedure online-snapshot is initializing 2016-01-11 19:21:00,350 INFO [localhost:58695.activeMasterManager] procedure.ZKProcedureUtil(270): Clearing all procedure znodes: /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2016-01-11 19:21:00,351 DEBUG [RS:1;localhost:58702] zookeeper.RecoverableZooKeeper(594): Node /hbase/online-snapshot already exists 2016-01-11 19:21:00,352 DEBUG [localhost:58695.activeMasterManager] procedure.ZKProcedureCoordinatorRpcs(248): Starting the controller for procedure member:localhost,58695,1452568859727 2016-01-11 19:21:00,352 DEBUG [RS:1;localhost:58702] zookeeper.RecoverableZooKeeper(594): Node /hbase/online-snapshot/acquired already exists 2016-01-11 19:21:00,354 DEBUG [RS:1;localhost:58702] zookeeper.RecoverableZooKeeper(594): Node /hbase/online-snapshot/abort already exists 2016-01-11 19:21:00,355 DEBUG [RS:0;localhost:58697] procedure.RegionServerProcedureManagerHost(46): Procedure online-snapshot is initialized 2016-01-11 19:21:00,355 DEBUG [RS:1;localhost:58702] procedure.RegionServerProcedureManagerHost(46): Procedure online-snapshot is initialized 2016-01-11 19:21:00,356 DEBUG [localhost:58695.activeMasterManager] zookeeper.RecoverableZooKeeper(594): Node /hbase/online-snapshot/acquired already exists 2016-01-11 19:21:00,357 INFO [localhost:58695.activeMasterManager] procedure.ZKProcedureUtil(270): Clearing all procedure znodes: /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2016-01-11 19:21:00,358 DEBUG [localhost:58695.activeMasterManager] procedure.ZKProcedureCoordinatorRpcs(248): Starting the controller for procedure member:localhost,58695,1452568859727 2016-01-11 19:21:00,359 INFO [RS:0;localhost:58697] regionserver.MemStoreFlusher(125): globalMemStoreLimit=995.6 M, globalMemStoreLimitLowMark=945.8 M, maxHeap=2.4 G 2016-01-11 19:21:00,359 INFO [RS:1;localhost:58702] regionserver.MemStoreFlusher(125): globalMemStoreLimit=995.6 M, globalMemStoreLimitLowMark=945.8 M, maxHeap=2.4 G 2016-01-11 19:21:00,366 INFO [RS:0;localhost:58697] regionserver.HRegionServer$CompactionChecker(1523): CompactionChecker runs every 1sec 2016-01-11 19:21:00,366 INFO [RS:1;localhost:58702] regionserver.HRegionServer$CompactionChecker(1523): CompactionChecker runs every 1sec 2016-01-11 19:21:00,370 DEBUG [RS:0;localhost:58697] ipc.AbstractRpcClient(115): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@36b38f2d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=localhost/127.0.0.1:0 2016-01-11 19:21:00,370 DEBUG [RS:1;localhost:58702] ipc.AbstractRpcClient(115): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4509bd1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=localhost/127.0.0.1:0 2016-01-11 19:21:00,374 DEBUG [RS:0;localhost:58697] regionserver.ShutdownHook(87): Installed shutdown hook thread: Shutdownhook:RS:0;localhost:58697 2016-01-11 19:21:00,374 DEBUG [RS:1;localhost:58702] regionserver.ShutdownHook(87): Installed shutdown hook thread: Shutdownhook:RS:1;localhost:58702 2016-01-11 19:21:00,386 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2016-01-11 19:21:00,386 DEBUG [RS:1;localhost:58702] zookeeper.ZKUtil(368): regionserver:58702-0x15233d9d0910002, quorum=localhost:54890, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/localhost,58702,1452568860027 2016-01-11 19:21:00,386 DEBUG [RS:0;localhost:58697] zookeeper.ZKUtil(368): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/localhost,58697,1452568859983 2016-01-11 19:21:00,387 DEBUG [main-EventThread] zookeeper.ZKUtil(368): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/localhost,58702,1452568860027 2016-01-11 19:21:00,387 DEBUG [main-EventThread] zookeeper.ZKUtil(368): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/localhost,58697,1452568859983 2016-01-11 19:21:00,388 DEBUG [main-EventThread] zookeeper.RegionServerTracker(93): Added tracking of RS /hbase/rs/localhost,58702,1452568860027 2016-01-11 19:21:00,389 DEBUG [main-EventThread] zookeeper.RegionServerTracker(93): Added tracking of RS /hbase/rs/localhost,58697,1452568859983 2016-01-11 19:21:00,395 INFO [RS:1;localhost:58702] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2016-01-11 19:21:00,395 INFO [RS:0;localhost:58697] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2016-01-11 19:21:00,396 INFO [RS:0;localhost:58697] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2016-01-11 19:21:00,395 INFO [RS:1;localhost:58702] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2016-01-11 19:21:00,397 INFO [RS:1;localhost:58702] regionserver.HRegionServer(2276): reportForDuty to master=localhost,58695,1452568859727 with port=58702, startcode=1452568860027 2016-01-11 19:21:00,397 INFO [RS:0;localhost:58697] regionserver.HRegionServer(2276): reportForDuty to master=localhost,58695,1452568859727 with port=58697, startcode=1452568859983 2016-01-11 19:21:00,402 INFO [localhost:58695.activeMasterManager] master.MasterCoprocessorHost(86): System coprocessor loading is enabled 2016-01-11 19:21:00,403 DEBUG [localhost:58695.activeMasterManager] executor.ExecutorService(99): Starting executor service name=MASTER_OPEN_REGION-localhost:58695, corePoolSize=5, maxPoolSize=5 2016-01-11 19:21:00,404 DEBUG [localhost:58695.activeMasterManager] executor.ExecutorService(99): Starting executor service name=MASTER_CLOSE_REGION-localhost:58695, corePoolSize=5, maxPoolSize=5 2016-01-11 19:21:00,404 DEBUG [localhost:58695.activeMasterManager] executor.ExecutorService(99): Starting executor service name=MASTER_SERVER_OPERATIONS-localhost:58695, corePoolSize=5, maxPoolSize=5 2016-01-11 19:21:00,404 DEBUG [localhost:58695.activeMasterManager] executor.ExecutorService(99): Starting executor service name=MASTER_META_SERVER_OPERATIONS-localhost:58695, corePoolSize=5, maxPoolSize=5 2016-01-11 19:21:00,404 DEBUG [localhost:58695.activeMasterManager] executor.ExecutorService(99): Starting executor service name=M_LOG_REPLAY_OPS-localhost:58695, corePoolSize=10, maxPoolSize=10 2016-01-11 19:21:00,404 DEBUG [localhost:58695.activeMasterManager] executor.ExecutorService(99): Starting executor service name=MASTER_TABLE_OPERATIONS-localhost:58695, corePoolSize=1, maxPoolSize=1 2016-01-11 19:21:00,424 INFO [localhost:58695.activeMasterManager] procedure2.ProcedureExecutor(472): Starting procedure executor threads=9 2016-01-11 19:21:00,425 INFO [localhost:58695.activeMasterManager] wal.WALProcedureStore(251): Starting WAL Procedure Store lease recovery 2016-01-11 19:21:00,426 WARN [localhost:58695.activeMasterManager] wal.WALProcedureStore(879): Log directory not found: File hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/MasterProcWALs does not exist. 2016-01-11 19:21:00,434 DEBUG [localhost:58695.activeMasterManager] wal.WALProcedureStore(772): Roll new state log: 1 2016-01-11 19:21:00,435 INFO [localhost:58695.activeMasterManager] wal.WALProcedureStore(274): Lease acquired for flushLogId: 1 2016-01-11 19:21:00,436 DEBUG [localhost:58695.activeMasterManager] wal.WALProcedureStore(291): No state logs to replay. 2016-01-11 19:21:00,436 DEBUG [localhost:58695.activeMasterManager] procedure2.ProcedureExecutor$1(298): load procedures maxProcId=0 2016-01-11 19:21:00,442 DEBUG [localhost:58695.activeMasterManager] cleaner.CleanerChore(91): initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2016-01-11 19:21:00,444 INFO [localhost:58695.activeMasterManager] zookeeper.RecoverableZooKeeper(120): Process identifier=replicationLogCleaner connecting to ZooKeeper ensemble=localhost:54890 2016-01-11 19:21:00,447 DEBUG [localhost:58695.activeMasterManager-EventThread] zookeeper.ZooKeeperWatcher(554): replicationLogCleaner0x0, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2016-01-11 19:21:00,449 DEBUG [localhost:58695.activeMasterManager-EventThread] zookeeper.ZooKeeperWatcher(638): replicationLogCleaner-0x15233d9d0910007 connected 2016-01-11 19:21:00,456 DEBUG [localhost:58695.activeMasterManager] cleaner.CleanerChore(91): initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2016-01-11 19:21:00,458 DEBUG [RS:1;localhost:58702] ipc.RpcClientImpl$Connection(351): Use SIMPLE authentication for service RegionServerStatusService, sasl=false 2016-01-11 19:21:00,458 DEBUG [RS:0;localhost:58697] ipc.RpcClientImpl$Connection(351): Use SIMPLE authentication for service RegionServerStatusService, sasl=false 2016-01-11 19:21:00,460 DEBUG [localhost:58695.activeMasterManager] cleaner.CleanerChore(91): initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotLogCleaner 2016-01-11 19:21:00,461 DEBUG [localhost:58695.activeMasterManager] cleaner.CleanerChore(91): initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2016-01-11 19:21:00,462 DEBUG [localhost:58695.activeMasterManager] cleaner.CleanerChore(91): initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2016-01-11 19:21:00,463 DEBUG [localhost:58695.activeMasterManager] cleaner.CleanerChore(91): initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2016-01-11 19:21:00,463 INFO [localhost:58695.activeMasterManager] master.ServerManager(1028): Waiting for region servers count to settle; currently checked in 0, slept for 0 ms, expecting minimum of 2, maximum of 2, timeout of 4500 ms, interval of 1500 ms. 2016-01-11 19:21:00,473 DEBUG [RS:1;localhost:58702] ipc.RpcClientImpl$Connection(716): Connecting to localhost/127.0.0.1:58695 2016-01-11 19:21:00,473 DEBUG [RS:0;localhost:58697] ipc.RpcClientImpl$Connection(716): Connecting to localhost/127.0.0.1:58695 2016-01-11 19:21:00,481 DEBUG [RpcServer.listener,port=58695] ipc.RpcServer$Listener(879): RpcServer.listener,port=58695: connection from 127.0.0.1:58713; # active connections: 1 2016-01-11 19:21:00,481 DEBUG [RpcServer.listener,port=58695] ipc.RpcServer$Listener(879): RpcServer.listener,port=58695: connection from 127.0.0.1:58714; # active connections: 2 2016-01-11 19:21:00,483 INFO [RpcServer.reader=2,bindAddress=localhost,port=58695] ipc.RpcServer$Connection(1712): Auth successful for elliott.hfs.0 (auth:SIMPLE) 2016-01-11 19:21:00,483 INFO [RpcServer.reader=1,bindAddress=localhost,port=58695] ipc.RpcServer$Connection(1712): Auth successful for elliott.hfs.1 (auth:SIMPLE) 2016-01-11 19:21:00,486 INFO [RpcServer.reader=2,bindAddress=localhost,port=58695] ipc.RpcServer$Connection(1742): Connection from 127.0.0.1 port: 58714 with version info: version: "1.2.0" url: "git://elliott-mbp/Users/elliott/Code/Public/hbase" revision: "4b59f62c231a39fd4a824c0cbf4c2166c4ec2a76" user: "elliott" date: "Mon Jan 11 19:20:24 PST 2016" src_checksum: "c2d37028b1462df7bac2e95df3bcc070" 2016-01-11 19:21:00,486 INFO [RpcServer.reader=1,bindAddress=localhost,port=58695] ipc.RpcServer$Connection(1742): Connection from 127.0.0.1 port: 58713 with version info: version: "1.2.0" url: "git://elliott-mbp/Users/elliott/Code/Public/hbase" revision: "4b59f62c231a39fd4a824c0cbf4c2166c4ec2a76" user: "elliott" date: "Mon Jan 11 19:20:24 PST 2016" src_checksum: "c2d37028b1462df7bac2e95df3bcc070" 2016-01-11 19:21:00,502 INFO [B.defaultRpcServer.handler=0,queue=0,port=58695] master.ServerManager(459): Registering server=localhost,58697,1452568859983 2016-01-11 19:21:00,502 INFO [B.defaultRpcServer.handler=4,queue=0,port=58695] master.ServerManager(459): Registering server=localhost,58702,1452568860027 2016-01-11 19:21:00,514 INFO [localhost:58695.activeMasterManager] master.ServerManager(1045): Finished waiting for region servers count to settle; checked in 2, slept for 51 ms, expecting minimum of 2, maximum of 2, master is running 2016-01-11 19:21:00,515 INFO [RS:0;localhost:58697] regionserver.HRegionServer(1359): Config from master: hbase.rootdir=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb 2016-01-11 19:21:00,515 INFO [RS:1;localhost:58702] regionserver.HRegionServer(1359): Config from master: hbase.rootdir=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb 2016-01-11 19:21:00,516 INFO [RS:0;localhost:58697] regionserver.HRegionServer(1359): Config from master: fs.defaultFS=hdfs://localhost:58683 2016-01-11 19:21:00,516 INFO [RS:0;localhost:58697] regionserver.HRegionServer(1359): Config from master: hbase.master.info.port=-1 2016-01-11 19:21:00,516 INFO [RS:1;localhost:58702] regionserver.HRegionServer(1359): Config from master: fs.defaultFS=hdfs://localhost:58683 2016-01-11 19:21:00,516 WARN [RS:0;localhost:58697] hbase.ZNodeClearer(58): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2016-01-11 19:21:00,517 INFO [RS:1;localhost:58702] regionserver.HRegionServer(1359): Config from master: hbase.master.info.port=-1 2016-01-11 19:21:00,517 INFO [RS:0;localhost:58697] hfile.CacheConfig(292): blockCache=LruBlockCache{blockCount=0, currentSize=1071840, freeSize=1042890464, maxSize=1043962304, heapSize=1071840, minSize=991764160, minFactor=0.95, multiSize=495882080, multiFactor=0.5, singleSize=247941040, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 2016-01-11 19:21:00,517 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:00,517 WARN [RS:1;localhost:58702] hbase.ZNodeClearer(58): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2016-01-11 19:21:00,517 DEBUG [RS:0;localhost:58697] regionserver.HRegionServer(1622): logdir=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/WALs/localhost,58697,1452568859983 2016-01-11 19:21:00,517 INFO [RS:1;localhost:58702] hfile.CacheConfig(292): blockCache=LruBlockCache{blockCount=0, currentSize=1071840, freeSize=1042890464, maxSize=1043962304, heapSize=1071840, minSize=991764160, minFactor=0.95, multiSize=495882080, multiFactor=0.5, singleSize=247941040, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 2016-01-11 19:21:00,518 DEBUG [RS:1;localhost:58702] regionserver.HRegionServer(1622): logdir=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/WALs/localhost,58702,1452568860027 2016-01-11 19:21:00,524 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:00,528 DEBUG [RS:0;localhost:58697] zookeeper.RecoverableZooKeeper(594): Node /hbase/replication/peers already exists 2016-01-11 19:21:00,528 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:00,533 DEBUG [RS:0;localhost:58697] regionserver.Replication(143): ReplicationStatisticsThread 300 2016-01-11 19:21:00,533 DEBUG [RS:1;localhost:58702] regionserver.Replication(143): ReplicationStatisticsThread 300 2016-01-11 19:21:00,545 INFO [RS:0;localhost:58697] wal.WALFactory(144): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.DefaultWALProvider 2016-01-11 19:21:00,545 INFO [RS:1;localhost:58702] wal.WALFactory(144): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.DefaultWALProvider 2016-01-11 19:21:00,560 INFO [RS:1;localhost:58702] wal.FSHLog(529): WAL configuration: blocksize=128 MB, rollsize=121.60 MB, prefix=localhost%2C58702%2C1452568860027.default, suffix=, logDir=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/WALs/localhost,58702,1452568860027, archiveDir=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/oldWALs 2016-01-11 19:21:00,560 INFO [RS:0;localhost:58697] wal.FSHLog(529): WAL configuration: blocksize=128 MB, rollsize=121.60 MB, prefix=localhost%2C58697%2C1452568859983.default, suffix=, logDir=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/WALs/localhost,58697,1452568859983, archiveDir=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/oldWALs 2016-01-11 19:21:00,598 INFO [RS:1;localhost:58702] wal.FSHLog(1397): Slow sync cost: 21 ms, current pipeline: [] 2016-01-11 19:21:00,598 INFO [RS:0;localhost:58697] wal.FSHLog(1397): Slow sync cost: 21 ms, current pipeline: [] 2016-01-11 19:21:00,598 INFO [RS:1;localhost:58702] wal.FSHLog(876): New WAL /user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/WALs/localhost,58702,1452568860027/localhost%2C58702%2C1452568860027.default.1452568860561 2016-01-11 19:21:00,599 INFO [RS:0;localhost:58697] wal.FSHLog(876): New WAL /user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/WALs/localhost,58697,1452568859983/localhost%2C58697%2C1452568859983.default.1452568860561 2016-01-11 19:21:00,628 INFO [RS:1;localhost:58702] regionserver.MetricsRegionServerWrapperImpl(101): Computing regionserver metrics every 5000 milliseconds 2016-01-11 19:21:00,628 INFO [RS:0;localhost:58697] regionserver.MetricsRegionServerWrapperImpl(101): Computing regionserver metrics every 5000 milliseconds 2016-01-11 19:21:00,635 DEBUG [RS:1;localhost:58702] executor.ExecutorService(99): Starting executor service name=RS_OPEN_REGION-localhost:58702, corePoolSize=3, maxPoolSize=3 2016-01-11 19:21:00,640 DEBUG [RS:1;localhost:58702] executor.ExecutorService(99): Starting executor service name=RS_OPEN_META-localhost:58702, corePoolSize=1, maxPoolSize=1 2016-01-11 19:21:00,643 DEBUG [RS:1;localhost:58702] executor.ExecutorService(99): Starting executor service name=RS_CLOSE_REGION-localhost:58702, corePoolSize=3, maxPoolSize=3 2016-01-11 19:21:00,643 DEBUG [RS:1;localhost:58702] executor.ExecutorService(99): Starting executor service name=RS_CLOSE_META-localhost:58702, corePoolSize=1, maxPoolSize=1 2016-01-11 19:21:00,643 DEBUG [RS:1;localhost:58702] executor.ExecutorService(99): Starting executor service name=RS_LOG_REPLAY_OPS-localhost:58702, corePoolSize=2, maxPoolSize=2 2016-01-11 19:21:00,643 DEBUG [RS:0;localhost:58697] executor.ExecutorService(99): Starting executor service name=RS_OPEN_REGION-localhost:58697, corePoolSize=3, maxPoolSize=3 2016-01-11 19:21:00,643 DEBUG [RS:1;localhost:58702] executor.ExecutorService(99): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-localhost:58702, corePoolSize=3, maxPoolSize=3 2016-01-11 19:21:00,643 DEBUG [RS:0;localhost:58697] executor.ExecutorService(99): Starting executor service name=RS_OPEN_META-localhost:58697, corePoolSize=1, maxPoolSize=1 2016-01-11 19:21:00,644 DEBUG [RS:0;localhost:58697] executor.ExecutorService(99): Starting executor service name=RS_CLOSE_REGION-localhost:58697, corePoolSize=3, maxPoolSize=3 2016-01-11 19:21:00,644 DEBUG [RS:0;localhost:58697] executor.ExecutorService(99): Starting executor service name=RS_CLOSE_META-localhost:58697, corePoolSize=1, maxPoolSize=1 2016-01-11 19:21:00,644 DEBUG [RS:0;localhost:58697] executor.ExecutorService(99): Starting executor service name=RS_LOG_REPLAY_OPS-localhost:58697, corePoolSize=2, maxPoolSize=2 2016-01-11 19:21:00,644 DEBUG [RS:0;localhost:58697] executor.ExecutorService(99): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-localhost:58697, corePoolSize=3, maxPoolSize=3 2016-01-11 19:21:00,646 DEBUG [RS:1;localhost:58702] zookeeper.ZKUtil(368): regionserver:58702-0x15233d9d0910002, quorum=localhost:54890, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/localhost,58702,1452568860027 2016-01-11 19:21:00,647 DEBUG [RS:0;localhost:58697] zookeeper.ZKUtil(368): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/localhost,58702,1452568860027 2016-01-11 19:21:00,647 DEBUG [RS:1;localhost:58702] zookeeper.ZKUtil(368): regionserver:58702-0x15233d9d0910002, quorum=localhost:54890, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/localhost,58697,1452568859983 2016-01-11 19:21:00,647 DEBUG [RS:0;localhost:58697] zookeeper.ZKUtil(368): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/localhost,58697,1452568859983 2016-01-11 19:21:00,647 INFO [RS:1;localhost:58702] regionserver.ReplicationSourceManager(236): Current list of replicators: [localhost,58702,1452568860027, localhost,58697,1452568859983] other RSs: [localhost,58702,1452568860027, localhost,58697,1452568859983] 2016-01-11 19:21:00,647 INFO [RS:0;localhost:58697] regionserver.ReplicationSourceManager(236): Current list of replicators: [localhost,58702,1452568860027, localhost,58697,1452568859983] other RSs: [localhost,58702,1452568860027, localhost,58697,1452568859983] 2016-01-11 19:21:00,698 INFO [SplitLogWorker-localhost:58697] regionserver.SplitLogWorker(134): SplitLogWorker localhost,58697,1452568859983 starting 2016-01-11 19:21:00,698 INFO [SplitLogWorker-localhost:58702] regionserver.SplitLogWorker(134): SplitLogWorker localhost,58702,1452568860027 starting 2016-01-11 19:21:00,701 INFO [RS:0;localhost:58697] regionserver.HeapMemoryManager(191): Starting HeapMemoryTuner chore. 2016-01-11 19:21:00,701 INFO [RS:1;localhost:58702] regionserver.HeapMemoryManager(191): Starting HeapMemoryTuner chore. 2016-01-11 19:21:00,705 INFO [RS:0;localhost:58697] regionserver.HRegionServer(1381): Serving as localhost,58697,1452568859983, RpcServer on localhost/127.0.0.1:58697, sessionid=0x15233d9d0910001 2016-01-11 19:21:00,705 INFO [RS:1;localhost:58702] regionserver.HRegionServer(1381): Serving as localhost,58702,1452568860027, RpcServer on localhost/127.0.0.1:58702, sessionid=0x15233d9d0910002 2016-01-11 19:21:00,705 DEBUG [RS:0;localhost:58697] procedure.RegionServerProcedureManagerHost(52): Procedure flush-table-proc is starting 2016-01-11 19:21:00,705 DEBUG [RS:1;localhost:58702] procedure.RegionServerProcedureManagerHost(52): Procedure flush-table-proc is starting 2016-01-11 19:21:00,705 DEBUG [RS:0;localhost:58697] flush.RegionServerFlushTableProcedureManager(103): Start region server flush procedure manager localhost,58697,1452568859983 2016-01-11 19:21:00,705 DEBUG [RS:1;localhost:58702] flush.RegionServerFlushTableProcedureManager(103): Start region server flush procedure manager localhost,58702,1452568860027 2016-01-11 19:21:00,705 DEBUG [RS:0;localhost:58697] procedure.ZKProcedureMemberRpcs(352): Starting procedure member 'localhost,58697,1452568859983' 2016-01-11 19:21:00,705 DEBUG [RS:1;localhost:58702] procedure.ZKProcedureMemberRpcs(352): Starting procedure member 'localhost,58702,1452568860027' 2016-01-11 19:21:00,706 DEBUG [RS:1;localhost:58702] procedure.ZKProcedureMemberRpcs(136): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2016-01-11 19:21:00,706 DEBUG [RS:0;localhost:58697] procedure.ZKProcedureMemberRpcs(136): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2016-01-11 19:21:00,706 DEBUG [RS:1;localhost:58702] procedure.ZKProcedureMemberRpcs(152): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2016-01-11 19:21:00,706 DEBUG [RS:0;localhost:58697] procedure.ZKProcedureMemberRpcs(152): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2016-01-11 19:21:00,707 DEBUG [RS:1;localhost:58702] procedure.RegionServerProcedureManagerHost(54): Procedure flush-table-proc is started 2016-01-11 19:21:00,707 DEBUG [RS:1;localhost:58702] procedure.RegionServerProcedureManagerHost(52): Procedure online-snapshot is starting 2016-01-11 19:21:00,707 DEBUG [RS:1;localhost:58702] snapshot.RegionServerSnapshotManager(124): Start Snapshot Manager localhost,58702,1452568860027 2016-01-11 19:21:00,707 DEBUG [RS:0;localhost:58697] procedure.RegionServerProcedureManagerHost(54): Procedure flush-table-proc is started 2016-01-11 19:21:00,707 DEBUG [RS:0;localhost:58697] procedure.RegionServerProcedureManagerHost(52): Procedure online-snapshot is starting 2016-01-11 19:21:00,707 DEBUG [RS:1;localhost:58702] procedure.ZKProcedureMemberRpcs(352): Starting procedure member 'localhost,58702,1452568860027' 2016-01-11 19:21:00,708 DEBUG [RS:1;localhost:58702] procedure.ZKProcedureMemberRpcs(136): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2016-01-11 19:21:00,707 DEBUG [RS:0;localhost:58697] snapshot.RegionServerSnapshotManager(124): Start Snapshot Manager localhost,58697,1452568859983 2016-01-11 19:21:00,708 DEBUG [RS:0;localhost:58697] procedure.ZKProcedureMemberRpcs(352): Starting procedure member 'localhost,58697,1452568859983' 2016-01-11 19:21:00,708 DEBUG [RS:0;localhost:58697] procedure.ZKProcedureMemberRpcs(136): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2016-01-11 19:21:00,708 DEBUG [RS:1;localhost:58702] procedure.ZKProcedureMemberRpcs(152): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2016-01-11 19:21:00,708 DEBUG [RS:0;localhost:58697] procedure.ZKProcedureMemberRpcs(152): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2016-01-11 19:21:00,708 DEBUG [RS:1;localhost:58702] procedure.RegionServerProcedureManagerHost(54): Procedure online-snapshot is started 2016-01-11 19:21:00,709 DEBUG [RS:0;localhost:58697] procedure.RegionServerProcedureManagerHost(54): Procedure online-snapshot is started 2016-01-11 19:21:00,718 INFO [RS:1;localhost:58702] quotas.RegionServerQuotaManager(53): Quota support disabled 2016-01-11 19:21:00,718 INFO [RS:0;localhost:58697] quotas.RegionServerQuotaManager(53): Quota support disabled 2016-01-11 19:21:00,733 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:00,935 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,140 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,344 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,345 INFO [localhost:58695.activeMasterManager] zookeeper.MetaTableLocator(525): Deleting hbase:meta region location in ZooKeeper 2016-01-11 19:21:01,345 DEBUG [localhost:58695.activeMasterManager] zookeeper.RecoverableZooKeeper(188): Node /hbase/meta-region-server already deleted, retry=false 2016-01-11 19:21:01,352 DEBUG [localhost:58695.activeMasterManager] balancer.RegionLocationFinder(269): HDFSBlocksDistribution not found in cache for region hbase:meta,,1.1588230740 2016-01-11 19:21:01,356 DEBUG [localhost:58695.activeMasterManager] regionserver.HRegionFileSystem(201): No StoreFiles for: hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/meta/1588230740/info 2016-01-11 19:21:01,360 DEBUG [localhost:58695.activeMasterManager] master.AssignmentManager(2454): No previous transition plan found (or ignoring an existing plan) for hbase:meta,,1.1588230740; generated random plan=hri=hbase:meta,,1.1588230740, src=, dest=localhost,58697,1452568859983; 2 (online=2) available servers, forceNewPlan=false 2016-01-11 19:21:01,360 INFO [localhost:58695.activeMasterManager] master.AssignmentManager(2100): Setting node as OFFLINED in ZooKeeper for region {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2016-01-11 19:21:01,360 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKAssign(205): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Creating (or updating) unassigned node 1588230740 with OFFLINE state 2016-01-11 19:21:01,367 DEBUG [localhost:58695.activeMasterManager] master.AssignmentManager(2117): Setting table hbase:meta to ENABLED state. 2016-01-11 19:21:01,367 INFO [localhost:58695.activeMasterManager] zookeeper.ZKTableStateManager(100): Moving table hbase:meta state from null to ENABLED 2016-01-11 19:21:01,372 INFO [localhost:58695.activeMasterManager] master.AssignmentManager(2132): Assigning hbase:meta,,1.1588230740 to localhost,58697,1452568859983 2016-01-11 19:21:01,373 INFO [localhost:58695.activeMasterManager] master.RegionStates(1109): Transition {1588230740 state=OFFLINE, ts=1452568861360, server=null} to {1588230740 state=PENDING_OPEN, ts=1452568861373, server=localhost,58697,1452568859983} 2016-01-11 19:21:01,373 DEBUG [localhost:58695.activeMasterManager] master.ServerManager(954): New admin connection to localhost,58697,1452568859983 2016-01-11 19:21:01,393 DEBUG [localhost:58695.activeMasterManager] ipc.RpcClientImpl$Connection(351): Use SIMPLE authentication for service AdminService, sasl=false 2016-01-11 19:21:01,394 DEBUG [localhost:58695.activeMasterManager] ipc.RpcClientImpl$Connection(716): Connecting to localhost/127.0.0.1:58697 2016-01-11 19:21:01,395 DEBUG [RpcServer.listener,port=58697] ipc.RpcServer$Listener(879): RpcServer.listener,port=58697: connection from 127.0.0.1:58721; # active connections: 1 2016-01-11 19:21:01,396 INFO [RpcServer.reader=1,bindAddress=localhost,port=58697] ipc.RpcServer$Connection(1712): Auth successful for elliott (auth:SIMPLE) 2016-01-11 19:21:01,396 INFO [RpcServer.reader=1,bindAddress=localhost,port=58697] ipc.RpcServer$Connection(1742): Connection from 127.0.0.1 port: 58721 with version info: version: "1.2.0" url: "git://elliott-mbp/Users/elliott/Code/Public/hbase" revision: "4b59f62c231a39fd4a824c0cbf4c2166c4ec2a76" user: "elliott" date: "Mon Jan 11 19:20:24 PST 2016" src_checksum: "c2d37028b1462df7bac2e95df3bcc070" 2016-01-11 19:21:01,400 INFO [PriorityRpcServer.handler=0,queue=0,port=58697] regionserver.RSRpcServices(1557): Open hbase:meta,,1.1588230740 2016-01-11 19:21:01,403 DEBUG [RS_OPEN_META-localhost:58697-0] zookeeper.ZKAssign(805): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Transitioning 1588230740 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING 2016-01-11 19:21:01,405 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/1588230740 2016-01-11 19:21:01,405 DEBUG [RS_OPEN_META-localhost:58697-0] zookeeper.ZKAssign(880): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Transitioned node 1588230740 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING 2016-01-11 19:21:01,405 INFO [RS_OPEN_META-localhost:58697-0] wal.WALFactory(144): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.DefaultWALProvider 2016-01-11 19:21:01,407 INFO [localhost:58695.activeMasterManager] master.ServerManager(687): AssignmentManager hasn't finished failover cleanup; waiting 2016-01-11 19:21:01,407 DEBUG [AM.ZK.Worker-pool2-t1] master.AssignmentManager(956): Handling RS_ZK_REGION_OPENING, server=localhost,58697,1452568859983, region=1588230740, current_state={1588230740 state=PENDING_OPEN, ts=1452568861373, server=localhost,58697,1452568859983} 2016-01-11 19:21:01,408 INFO [AM.ZK.Worker-pool2-t1] master.RegionStates(1109): Transition {1588230740 state=PENDING_OPEN, ts=1452568861373, server=localhost,58697,1452568859983} to {1588230740 state=OPENING, ts=1452568861408, server=localhost,58697,1452568859983} 2016-01-11 19:21:01,408 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,410 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,410 INFO [RS_OPEN_META-localhost:58697-0] wal.FSHLog(529): WAL configuration: blocksize=128 MB, rollsize=121.60 MB, prefix=localhost%2C58697%2C1452568859983..meta, suffix=.meta, logDir=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/WALs/localhost,58697,1452568859983, archiveDir=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/oldWALs 2016-01-11 19:21:01,411 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,412 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,412 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,413 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,414 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,415 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,417 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,417 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,421 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,421 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,424 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,426 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,427 INFO [RS_OPEN_META-localhost:58697-0] wal.FSHLog(1397): Slow sync cost: 13 ms, current pipeline: [] 2016-01-11 19:21:01,427 INFO [RS_OPEN_META-localhost:58697-0] wal.FSHLog(876): New WAL /user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/WALs/localhost,58697,1452568859983/localhost%2C58697%2C1452568859983..meta.1452568861410.meta 2016-01-11 19:21:01,427 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,428 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,428 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,429 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,430 DEBUG [RS_OPEN_META-localhost:58697-0] regionserver.HRegion(6408): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2016-01-11 19:21:01,430 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,430 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,431 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,432 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,433 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,433 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,434 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,434 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,435 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,436 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,437 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,437 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,438 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,439 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,440 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,440 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,441 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,442 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,443 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,443 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,444 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,446 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,446 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,447 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,447 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,448 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,449 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,449 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,450 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,451 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,451 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,452 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,452 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,453 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,454 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,454 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,455 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,455 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,456 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,457 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,457 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,458 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,458 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,459 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,460 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,460 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,461 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,461 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,462 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,463 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,463 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,464 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,464 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,465 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,465 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,466 DEBUG [RS_OPEN_META-localhost:58697-0] coprocessor.CoprocessorHost(182): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2016-01-11 19:21:01,466 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,467 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,467 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,468 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,468 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,469 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,470 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,470 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,471 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,471 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,472 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,473 DEBUG [RS_OPEN_META-localhost:58697-0] regionserver.HRegion(7654): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2016-01-11 19:21:01,473 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,473 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,474 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,475 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,475 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,476 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,477 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,477 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,478 INFO [RS_OPEN_META-localhost:58697-0] regionserver.RegionCoprocessorHost(367): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2016-01-11 19:21:01,478 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,479 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,479 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,480 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,480 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,481 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,482 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,483 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,483 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,484 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,484 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,485 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,486 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,486 DEBUG [RS_OPEN_META-localhost:58697-0] regionserver.MetricsRegionSourceImpl(70): Creating new MetricsRegionSourceImpl for table meta 1588230740 2016-01-11 19:21:01,486 DEBUG [RS_OPEN_META-localhost:58697-0] regionserver.HRegion(733): Instantiated hbase:meta,,1.1588230740 2016-01-11 19:21:01,486 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,487 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,488 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,488 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,489 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,490 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,490 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,491 INFO [StoreOpener-1588230740-1] hfile.CacheConfig(292): blockCache=LruBlockCache{blockCount=0, currentSize=1071840, freeSize=1042890464, maxSize=1043962304, heapSize=1071840, minSize=991764160, minFactor=0.95, multiSize=495882080, multiFactor=0.5, singleSize=247941040, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 2016-01-11 19:21:01,491 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,491 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(104): size [134217728, 9223372036854775807, 9223372036854775807); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000 2016-01-11 19:21:01,492 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,493 DEBUG [StoreOpener-1588230740-1] regionserver.HRegionFileSystem(201): No StoreFiles for: hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/meta/1588230740/info 2016-01-11 19:21:01,493 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,495 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,495 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,496 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,497 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,497 DEBUG [RS_OPEN_META-localhost:58697-0] regionserver.HRegion(3890): Found 0 recovered edits file(s) under hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/meta/1588230740 2016-01-11 19:21:01,498 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,498 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,499 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,500 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,500 DEBUG [RS_OPEN_META-localhost:58697-0] regionserver.FlushLargeStoresPolicy(56): hbase.hregion.percolumnfamilyflush.size.lower.bound is not specified, use global config(16777216) instead 2016-01-11 19:21:01,500 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,501 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,502 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,503 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,503 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,504 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,505 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,505 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,505 DEBUG [RS_OPEN_META-localhost:58697-0] wal.WALSplitter(729): Wrote region seqId=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/meta/1588230740/recovered.edits/3.seqid to file, newSeqId=3, maxSeqId=2 2016-01-11 19:21:01,506 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,506 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,506 INFO [RS_OPEN_META-localhost:58697-0] regionserver.HRegion(868): Onlined 1588230740; next sequenceid=3 2016-01-11 19:21:01,507 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,507 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,508 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,509 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,509 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,510 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,510 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,511 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,512 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,512 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,513 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,513 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,514 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,515 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,515 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,516 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,516 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,517 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,517 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,518 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,518 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,519 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,520 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,520 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,521 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,521 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,522 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,522 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,523 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,524 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,524 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,525 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,526 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,527 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,529 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,530 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,531 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,531 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,532 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,534 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,534 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,535 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,535 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,536 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,537 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,537 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,538 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,538 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,542 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,542 DEBUG [RS_OPEN_META-localhost:58697-0] zookeeper.ZKAssign(643): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Attempting to retransition opening state of node 1588230740 2016-01-11 19:21:01,542 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,543 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,543 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,544 INFO [PostOpenDeployTasks:1588230740] regionserver.HRegionServer(1912): Post open deploy tasks for hbase:meta,,1.1588230740 2016-01-11 19:21:01,544 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,545 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,545 INFO [PostOpenDeployTasks:1588230740] zookeeper.MetaTableLocator(439): Setting hbase:meta region location in ZooKeeper as localhost,58697,1452568859983 2016-01-11 19:21:01,546 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,546 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,547 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,547 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,548 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,549 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,549 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,550 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,550 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,551 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,552 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,552 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,553 DEBUG [PostOpenDeployTasks:1588230740] zookeeper.MetaTableLocator(451): META region location doesn't exist, create it 2016-01-11 19:21:01,553 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error) 2016-01-11 19:21:01,554 DEBUG [PostOpenDeployTasks:1588230740] regionserver.HRegionServer(1946): Finished post open deploy task for hbase:meta,,1.1588230740 2016-01-11 19:21:01,554 DEBUG [RS_OPEN_META-localhost:58697-0] zookeeper.ZKAssign(805): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Transitioning 1588230740 from RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENED 2016-01-11 19:21:01,556 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/1588230740 2016-01-11 19:21:01,556 DEBUG [RS_OPEN_META-localhost:58697-0] zookeeper.ZKAssign(880): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Transitioned node 1588230740 from RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENED 2016-01-11 19:21:01,556 DEBUG [RS_OPEN_META-localhost:58697-0] coordination.ZkOpenRegionCoordination(91): Transitioned 1588230740 to OPENED in zk on localhost,58697,1452568859983 2016-01-11 19:21:01,556 DEBUG [RS_OPEN_META-localhost:58697-0] handler.OpenRegionHandler(170): Opened hbase:meta,,1.1588230740 on localhost,58697,1452568859983 2016-01-11 19:21:01,557 DEBUG [AM.ZK.Worker-pool2-t2] master.AssignmentManager(956): Handling RS_ZK_REGION_OPENED, server=localhost,58697,1452568859983, region=1588230740, current_state={1588230740 state=OPENING, ts=1452568861408, server=localhost,58697,1452568859983} 2016-01-11 19:21:01,557 INFO [AM.ZK.Worker-pool2-t2] master.RegionStates(1109): Transition {1588230740 state=OPENING, ts=1452568861408, server=localhost,58697,1452568859983} to {1588230740 state=OPEN, ts=1452568861557, server=localhost,58697,1452568859983} 2016-01-11 19:21:01,558 INFO [AM.ZK.Worker-pool2-t2] coordination.ZkOpenRegionCoordination(348): Handling OPENED of 1588230740 from localhost,58695,1452568859727; deleting unassigned node 2016-01-11 19:21:01,560 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/region-in-transition/1588230740 2016-01-11 19:21:01,560 DEBUG [AM.ZK.Worker-pool2-t2] zookeeper.ZKAssign(479): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Deleted unassigned node 1588230740 in expected state RS_ZK_REGION_OPENED 2016-01-11 19:21:01,561 DEBUG [AM.ZK.Worker-pool2-t3] master.AssignmentManager$4(1345): Znode hbase:meta,,1.1588230740 deleted, state: {1588230740 state=OPEN, ts=1452568861557, server=localhost,58697,1452568859983} 2016-01-11 19:21:01,562 DEBUG [AM.ZK.Worker-pool2-t3] master.RegionStates(456): Onlined 1588230740 on localhost,58697,1452568859983 2016-01-11 19:21:01,563 INFO [localhost:58695.activeMasterManager] master.HMaster(944): hbase:meta with replicaId 0 assigned=1, rit=false, location=localhost,58697,1452568859983 2016-01-11 19:21:01,617 DEBUG [hconnection-0x5fbed0e5-shared--pool8-t1] ipc.RpcClientImpl$Connection(351): Use SIMPLE authentication for service ClientService, sasl=false 2016-01-11 19:21:01,617 DEBUG [hconnection-0x5fbed0e5-shared--pool8-t1] ipc.RpcClientImpl$Connection(716): Connecting to localhost/127.0.0.1:58697 2016-01-11 19:21:01,618 DEBUG [RpcServer.listener,port=58697] ipc.RpcServer$Listener(879): RpcServer.listener,port=58697: connection from 127.0.0.1:58725; # active connections: 2 2016-01-11 19:21:01,620 INFO [RpcServer.reader=2,bindAddress=localhost,port=58697] ipc.RpcServer$Connection(1712): Auth successful for elliott (auth:SIMPLE) 2016-01-11 19:21:01,620 INFO [RpcServer.reader=2,bindAddress=localhost,port=58697] ipc.RpcServer$Connection(1742): Connection from 127.0.0.1 port: 58725 with version info: version: "1.2.0" url: "git://elliott-mbp/Users/elliott/Code/Public/hbase" revision: "4b59f62c231a39fd4a824c0cbf4c2166c4ec2a76" user: "elliott" date: "Mon Jan 11 19:20:24 PST 2016" src_checksum: "c2d37028b1462df7bac2e95df3bcc070" 2016-01-11 19:21:01,667 INFO [localhost:58695.activeMasterManager] hbase.MetaMigrationConvertingToPB(165): hbase:meta doesn't have any entries to update. 2016-01-11 19:21:01,667 INFO [localhost:58695.activeMasterManager] hbase.MetaMigrationConvertingToPB(131): META already up-to date with PB serialization 2016-01-11 19:21:01,673 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKAssign(497): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Deleting any existing unassigned nodes 2016-01-11 19:21:01,674 INFO [localhost:58695.activeMasterManager] master.AssignmentManager(644): Clean cluster startup. Assigning user regions 2016-01-11 19:21:01,675 INFO [localhost:58695.activeMasterManager] master.AssignmentManager(502): Joined the cluster in 8ms, failover=false 2016-01-11 19:21:01,678 DEBUG [region-location-0] regionserver.HRegionFileSystem(201): No StoreFiles for: hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/meta/1588230740/info 2016-01-11 19:21:01,684 INFO [localhost:58695.activeMasterManager] master.TableNamespaceManager(90): Namespace table not found. Creating... 2016-01-11 19:21:01,837 DEBUG [localhost:58695.activeMasterManager] procedure2.ProcedureExecutor(654): Procedure CreateTableProcedure (table=hbase:namespace) id=1 owner=elliott state=RUNNABLE:CREATE_TABLE_PRE_OPERATION added to the store. 2016-01-11 19:21:01,852 DEBUG [ProcedureExecutor-0] lock.ZKInterProcessLockBase(226): Acquired a lock for /hbase/table-lock/hbase:namespace/write-master:586950000000000 2016-01-11 19:21:01,984 INFO [IPC Server handler 4 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58687 is added to blk_1073741833_1009{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6ab76dd9-6c4e-41f4-9b47-7b827e25bec7:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-70606f4d-e6bd-4292-9765-171d84a1305c:NORMAL|RBW]]} size 0 2016-01-11 19:21:01,986 INFO [IPC Server handler 5 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58684 is added to blk_1073741833_1009{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6ab76dd9-6c4e-41f4-9b47-7b827e25bec7:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-70606f4d-e6bd-4292-9765-171d84a1305c:NORMAL|RBW]]} size 0 2016-01-11 19:21:01,988 DEBUG [ProcedureExecutor-0] util.FSTableDescriptors(656): Wrote descriptor into: hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2016-01-11 19:21:01,991 INFO [RegionOpenAndInitThread-hbase:namespace-1] regionserver.HRegion(6219): creating HRegion hbase:namespace HTD == 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', VERSIONS => '10', IN_MEMORY => 'true', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', CACHE_DATA_IN_L1 => 'true', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} RootDir = hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/.tmp Table name == hbase:namespace 2016-01-11 19:21:02,005 INFO [IPC Server handler 6 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58684 is added to blk_1073741834_1010{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-222d7346-39ee-452c-9ca0-e0cb04e5fd92:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-84b79f3b-7553-476e-b313-9382dc0bd9af:NORMAL|RBW]]} size 0 2016-01-11 19:21:02,006 INFO [IPC Server handler 7 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58687 is added to blk_1073741834_1010{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-222d7346-39ee-452c-9ca0-e0cb04e5fd92:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-84b79f3b-7553-476e-b313-9382dc0bd9af:NORMAL|RBW]]} size 0 2016-01-11 19:21:02,007 DEBUG [RegionOpenAndInitThread-hbase:namespace-1] regionserver.HRegion(733): Instantiated hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. 2016-01-11 19:21:02,007 DEBUG [RegionOpenAndInitThread-hbase:namespace-1] regionserver.HRegion(1389): Closing hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b.: disabling compactions & flushes 2016-01-11 19:21:02,007 DEBUG [RegionOpenAndInitThread-hbase:namespace-1] regionserver.HRegion(1416): Updates disabled for region hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. 2016-01-11 19:21:02,007 INFO [RegionOpenAndInitThread-hbase:namespace-1] regionserver.HRegion(1519): Closed hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. 2016-01-11 19:21:02,181 INFO [ProcedureExecutor-0] hbase.MetaTableAccessor(1199): Added 1 2016-01-11 19:21:02,286 INFO [ProcedureExecutor-0] zookeeper.ZKTableStateManager(100): Moving table hbase:namespace state from null to ENABLING 2016-01-11 19:21:02,290 DEBUG [ProcedureExecutor-0] balancer.RegionLocationFinder(269): HDFSBlocksDistribution not found in cache for region hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. 2016-01-11 19:21:02,297 DEBUG [ProcedureExecutor-0] regionserver.HRegionFileSystem(201): No StoreFiles for: hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/namespace/33d6151ed04c075d7291124431dccb8b/info 2016-01-11 19:21:02,298 INFO [ProcedureExecutor-0] master.AssignmentManager(1613): Assigning 1 region(s) to localhost,58697,1452568859983 2016-01-11 19:21:02,300 DEBUG [ProcedureExecutor-0] zookeeper.ZKAssign(174): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Async create of unassigned node 33d6151ed04c075d7291124431dccb8b with OFFLINE state 2016-01-11 19:21:02,300 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/region-in-transition 2016-01-11 19:21:02,302 DEBUG [main-EventThread] master.OfflineCallback(69): rs={33d6151ed04c075d7291124431dccb8b state=OFFLINE, ts=1452568862289, server=null}, server=localhost,58697,1452568859983 2016-01-11 19:21:02,302 DEBUG [main-EventThread] master.OfflineCallback$ExistCallback(106): rs={33d6151ed04c075d7291124431dccb8b state=OFFLINE, ts=1452568862289, server=null}, server=localhost,58697,1452568859983 2016-01-11 19:21:02,306 DEBUG [ProcedureExecutor-0] master.AssignmentManager(1666): localhost,58697,1452568859983 unassigned znodes=1 of total=1; oldCounter=0 2016-01-11 19:21:02,306 INFO [ProcedureExecutor-0] master.RegionStates(1109): Transition {33d6151ed04c075d7291124431dccb8b state=OFFLINE, ts=1452568862300, server=null} to {33d6151ed04c075d7291124431dccb8b state=PENDING_OPEN, ts=1452568862306, server=localhost,58697,1452568859983} 2016-01-11 19:21:02,308 INFO [PriorityRpcServer.handler=1,queue=1,port=58697] regionserver.RSRpcServices(1557): Open hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. 2016-01-11 19:21:02,312 DEBUG [RS_OPEN_REGION-localhost:58697-0] zookeeper.ZKAssign(805): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Transitioning 33d6151ed04c075d7291124431dccb8b from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING 2016-01-11 19:21:02,312 DEBUG [ProcedureExecutor-0] master.AssignmentManager(1808): Bulk assigning done for localhost,58697,1452568859983 2016-01-11 19:21:02,312 INFO [ProcedureExecutor-0] zookeeper.ZKTableStateManager(100): Moving table hbase:namespace state from ENABLING to ENABLED 2016-01-11 19:21:02,313 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/33d6151ed04c075d7291124431dccb8b 2016-01-11 19:21:02,313 DEBUG [RS_OPEN_REGION-localhost:58697-0] zookeeper.ZKAssign(880): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Transitioned node 33d6151ed04c075d7291124431dccb8b from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING 2016-01-11 19:21:02,313 DEBUG [RS_OPEN_REGION-localhost:58697-0] regionserver.HRegion(6408): Opening region: {ENCODED => 33d6151ed04c075d7291124431dccb8b, NAME => 'hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b.', STARTKEY => '', ENDKEY => ''} 2016-01-11 19:21:02,313 DEBUG [AM.ZK.Worker-pool2-t5] master.AssignmentManager(956): Handling RS_ZK_REGION_OPENING, server=localhost,58697,1452568859983, region=33d6151ed04c075d7291124431dccb8b, current_state={33d6151ed04c075d7291124431dccb8b state=PENDING_OPEN, ts=1452568862306, server=localhost,58697,1452568859983} 2016-01-11 19:21:02,314 INFO [AM.ZK.Worker-pool2-t5] master.RegionStates(1109): Transition {33d6151ed04c075d7291124431dccb8b state=PENDING_OPEN, ts=1452568862306, server=localhost,58697,1452568859983} to {33d6151ed04c075d7291124431dccb8b state=OPENING, ts=1452568862314, server=localhost,58697,1452568859983} 2016-01-11 19:21:02,314 DEBUG [RS_OPEN_REGION-localhost:58697-0] regionserver.MetricsRegionSourceImpl(70): Creating new MetricsRegionSourceImpl for table namespace 33d6151ed04c075d7291124431dccb8b 2016-01-11 19:21:02,314 DEBUG [RS_OPEN_REGION-localhost:58697-0] regionserver.HRegion(733): Instantiated hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. 2016-01-11 19:21:02,318 INFO [StoreOpener-33d6151ed04c075d7291124431dccb8b-1] hfile.CacheConfig(292): blockCache=LruBlockCache{blockCount=0, currentSize=1071840, freeSize=1042890464, maxSize=1043962304, heapSize=1071840, minSize=991764160, minFactor=0.95, multiSize=495882080, multiFactor=0.5, singleSize=247941040, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 2016-01-11 19:21:02,319 INFO [StoreOpener-33d6151ed04c075d7291124431dccb8b-1] compactions.CompactionConfiguration(104): size [134217728, 9223372036854775807, 9223372036854775807); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000 2016-01-11 19:21:02,320 DEBUG [StoreOpener-33d6151ed04c075d7291124431dccb8b-1] regionserver.HRegionFileSystem(201): No StoreFiles for: hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/namespace/33d6151ed04c075d7291124431dccb8b/info 2016-01-11 19:21:02,321 DEBUG [RS_OPEN_REGION-localhost:58697-0] regionserver.HRegion(3890): Found 0 recovered edits file(s) under hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/namespace/33d6151ed04c075d7291124431dccb8b 2016-01-11 19:21:02,323 DEBUG [RS_OPEN_REGION-localhost:58697-0] regionserver.FlushLargeStoresPolicy(56): hbase.hregion.percolumnfamilyflush.size.lower.bound is not specified, use global config(16777216) instead 2016-01-11 19:21:02,325 DEBUG [RS_OPEN_REGION-localhost:58697-0] wal.WALSplitter(729): Wrote region seqId=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/namespace/33d6151ed04c075d7291124431dccb8b/recovered.edits/2.seqid to file, newSeqId=2, maxSeqId=0 2016-01-11 19:21:02,325 INFO [RS_OPEN_REGION-localhost:58697-0] regionserver.HRegion(868): Onlined 33d6151ed04c075d7291124431dccb8b; next sequenceid=2 2016-01-11 19:21:02,329 DEBUG [RS_OPEN_REGION-localhost:58697-0] zookeeper.ZKAssign(643): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Attempting to retransition opening state of node 33d6151ed04c075d7291124431dccb8b 2016-01-11 19:21:02,329 INFO [PostOpenDeployTasks:33d6151ed04c075d7291124431dccb8b] regionserver.HRegionServer(1912): Post open deploy tasks for hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. 2016-01-11 19:21:02,335 INFO [PostOpenDeployTasks:33d6151ed04c075d7291124431dccb8b] hbase.MetaTableAccessor(1394): Updated row hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. with server=localhost,58697,1452568859983 2016-01-11 19:21:02,335 DEBUG [PostOpenDeployTasks:33d6151ed04c075d7291124431dccb8b] regionserver.HRegionServer(1946): Finished post open deploy task for hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. 2016-01-11 19:21:02,336 DEBUG [RS_OPEN_REGION-localhost:58697-0] zookeeper.ZKAssign(805): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Transitioning 33d6151ed04c075d7291124431dccb8b from RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENED 2016-01-11 19:21:02,337 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/33d6151ed04c075d7291124431dccb8b 2016-01-11 19:21:02,337 DEBUG [RS_OPEN_REGION-localhost:58697-0] zookeeper.ZKAssign(880): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Transitioned node 33d6151ed04c075d7291124431dccb8b from RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENED 2016-01-11 19:21:02,337 DEBUG [RS_OPEN_REGION-localhost:58697-0] coordination.ZkOpenRegionCoordination(91): Transitioned 33d6151ed04c075d7291124431dccb8b to OPENED in zk on localhost,58697,1452568859983 2016-01-11 19:21:02,337 DEBUG [RS_OPEN_REGION-localhost:58697-0] handler.OpenRegionHandler(170): Opened hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. on localhost,58697,1452568859983 2016-01-11 19:21:02,337 DEBUG [AM.ZK.Worker-pool2-t6] master.AssignmentManager(956): Handling RS_ZK_REGION_OPENED, server=localhost,58697,1452568859983, region=33d6151ed04c075d7291124431dccb8b, current_state={33d6151ed04c075d7291124431dccb8b state=OPENING, ts=1452568862314, server=localhost,58697,1452568859983} 2016-01-11 19:21:02,337 INFO [AM.ZK.Worker-pool2-t6] master.RegionStates(1109): Transition {33d6151ed04c075d7291124431dccb8b state=OPENING, ts=1452568862314, server=localhost,58697,1452568859983} to {33d6151ed04c075d7291124431dccb8b state=OPEN, ts=1452568862337, server=localhost,58697,1452568859983} 2016-01-11 19:21:02,337 DEBUG [AM.ZK.Worker-pool2-t6] coordination.ZkOpenRegionCoordination(350): Handling OPENED of 33d6151ed04c075d7291124431dccb8b from localhost,58695,1452568859727; deleting unassigned node 2016-01-11 19:21:02,338 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/region-in-transition/33d6151ed04c075d7291124431dccb8b 2016-01-11 19:21:02,338 DEBUG [AM.ZK.Worker-pool2-t6] zookeeper.ZKAssign(479): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Deleted unassigned node 33d6151ed04c075d7291124431dccb8b in expected state RS_ZK_REGION_OPENED 2016-01-11 19:21:02,339 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/region-in-transition 2016-01-11 19:21:02,340 DEBUG [AM.ZK.Worker-pool2-t8] master.AssignmentManager$4(1345): Znode hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. deleted, state: {33d6151ed04c075d7291124431dccb8b state=OPEN, ts=1452568862337, server=localhost,58697,1452568859983} 2016-01-11 19:21:02,340 DEBUG [AM.ZK.Worker-pool2-t8] master.RegionStates(456): Onlined 33d6151ed04c075d7291124431dccb8b on localhost,58697,1452568859983 2016-01-11 19:21:02,356 DEBUG [localhost:58695.activeMasterManager] zookeeper.ZKUtil(370): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2016-01-11 19:21:02,356 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2016-01-11 19:21:02,388 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2016-01-11 19:21:02,391 DEBUG [main-EventThread] hbase.ZKNamespaceManager(196): Updating namespace cache from node default with data: \x0A\x07default 2016-01-11 19:21:02,395 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2016-01-11 19:21:02,397 DEBUG [main-EventThread] hbase.ZKNamespaceManager(196): Updating namespace cache from node default with data: \x0A\x07default 2016-01-11 19:21:02,397 DEBUG [main-EventThread] hbase.ZKNamespaceManager(196): Updating namespace cache from node hbase with data: \x0A\x05hbase 2016-01-11 19:21:02,401 DEBUG [localhost:58695.activeMasterManager] zookeeper.RecoverableZooKeeper(594): Node /hbase/namespace/default already exists 2016-01-11 19:21:02,402 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2016-01-11 19:21:02,402 DEBUG [localhost:58695.activeMasterManager] zookeeper.RecoverableZooKeeper(594): Node /hbase/namespace/hbase already exists 2016-01-11 19:21:02,404 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2016-01-11 19:21:02,404 INFO [localhost:58695.activeMasterManager] master.HMaster(775): Master has completed initialization 2016-01-11 19:21:02,406 INFO [localhost:58695.activeMasterManager] quotas.MasterQuotaManager(62): Quota support disabled 2016-01-11 19:21:02,410 INFO [main] zookeeper.RecoverableZooKeeper(120): Process identifier=hconnection-0x72fe8a4f connecting to ZooKeeper ensemble=localhost:54890 2016-01-11 19:21:02,412 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): hconnection-0x72fe8a4f0x0, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2016-01-11 19:21:02,413 DEBUG [main] ipc.AbstractRpcClient(115): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@43effd89, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2016-01-11 19:21:02,414 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(638): hconnection-0x72fe8a4f-0x15233d9d0910008 connected 2016-01-11 19:21:02,414 DEBUG [htable-pool12-t1] ipc.RpcClientImpl$Connection(351): Use SIMPLE authentication for service ClientService, sasl=false 2016-01-11 19:21:02,414 DEBUG [htable-pool12-t1] ipc.RpcClientImpl$Connection(716): Connecting to localhost/127.0.0.1:58697 2016-01-11 19:21:02,415 DEBUG [RpcServer.listener,port=58697] ipc.RpcServer$Listener(879): RpcServer.listener,port=58697: connection from 127.0.0.1:58734; # active connections: 3 2016-01-11 19:21:02,416 INFO [RpcServer.reader=0,bindAddress=localhost,port=58697] ipc.RpcServer$Connection(1712): Auth successful for elliott (auth:SIMPLE) 2016-01-11 19:21:02,416 INFO [RpcServer.reader=0,bindAddress=localhost,port=58697] ipc.RpcServer$Connection(1742): Connection from 127.0.0.1 port: 58734 with version info: version: "1.2.0" url: "git://elliott-mbp/Users/elliott/Code/Public/hbase" revision: "4b59f62c231a39fd4a824c0cbf4c2166c4ec2a76" user: "elliott" date: "Mon Jan 11 19:20:24 PST 2016" src_checksum: "c2d37028b1462df7bac2e95df3bcc070" 2016-01-11 19:21:02,420 INFO [main] client.ConnectionManager$HConnectionImplementation(1712): Closing zookeeper sessionid=0x15233d9d0910008 2016-01-11 19:21:02,421 DEBUG [main] ipc.RpcClientImpl(1147): Stopping rpc client 2016-01-11 19:21:02,422 DEBUG [RpcServer.reader=0,bindAddress=localhost,port=58697] ipc.RpcServer$Listener(911): RpcServer.listener,port=58697: DISCONNECTING client 127.0.0.1:58734 because read count=-1. Number of active connections: 3 2016-01-11 19:21:02,454 INFO [main] zookeeper.RecoverableZooKeeper(120): Process identifier=hconnection-0x2227a6c1 connecting to ZooKeeper ensemble=localhost:54890 2016-01-11 19:21:02,457 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): hconnection-0x2227a6c10x0, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2016-01-11 19:21:02,457 DEBUG [main] ipc.AbstractRpcClient(115): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1e9804b9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2016-01-11 19:21:02,458 INFO [main] hbase.HBaseTestingUtility(1043): Minicluster is up 2016-01-11 19:21:02,458 INFO [main] hbase.HBaseTestingUtility(1211): The hbase.fs.tmp.dir is set to /user/elliott/hbase-staging 2016-01-11 19:21:02,458 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(638): hconnection-0x2227a6c1-0x15233d9d0910009 connected 2016-01-11 19:21:02,465 DEBUG [main] ipc.RpcClientImpl$Connection(351): Use SIMPLE authentication for service MasterService, sasl=false 2016-01-11 19:21:02,465 DEBUG [main] ipc.RpcClientImpl$Connection(716): Connecting to localhost/127.0.0.1:58695 2016-01-11 19:21:02,467 DEBUG [RpcServer.listener,port=58695] ipc.RpcServer$Listener(879): RpcServer.listener,port=58695: connection from 127.0.0.1:58736; # active connections: 3 2016-01-11 19:21:02,467 INFO [RpcServer.reader=0,bindAddress=localhost,port=58695] ipc.RpcServer$Connection(1712): Auth successful for elliott (auth:SIMPLE) 2016-01-11 19:21:02,467 INFO [RpcServer.reader=0,bindAddress=localhost,port=58695] ipc.RpcServer$Connection(1742): Connection from 127.0.0.1 port: 58736 with version info: version: "1.2.0" url: "git://elliott-mbp/Users/elliott/Code/Public/hbase" revision: "4b59f62c231a39fd4a824c0cbf4c2166c4ec2a76" user: "elliott" date: "Mon Jan 11 19:20:24 PST 2016" src_checksum: "c2d37028b1462df7bac2e95df3bcc070" 2016-01-11 19:21:02,475 INFO [B.defaultRpcServer.handler=2,queue=0,port=58695] master.HMaster(2428): Client=elliott//127.0.0.1 creating {NAME => 'TestSplitWalDataLoss'} 2016-01-11 19:21:02,479 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2016-01-11 19:21:02,481 DEBUG [main-EventThread] hbase.ZKNamespaceManager(196): Updating namespace cache from node default with data: \x0A\x07default 2016-01-11 19:21:02,481 DEBUG [main-EventThread] hbase.ZKNamespaceManager(196): Updating namespace cache from node hbase with data: \x0A\x05hbase 2016-01-11 19:21:02,481 DEBUG [main-EventThread] hbase.ZKNamespaceManager(196): Updating namespace cache from node TestSplitWalDataLoss with data: \x0A\x14TestSplitWalDataLoss 2016-01-11 19:21:02,486 INFO [B.defaultRpcServer.handler=0,queue=0,port=58695] master.HMaster(1481): Client=elliott//127.0.0.1 create 'TestSplitWalDataLoss:dataloss', {NAME => 'f', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2016-01-11 19:21:02,520 DEBUG [B.defaultRpcServer.handler=0,queue=0,port=58695] procedure2.ProcedureExecutor(654): Procedure CreateTableProcedure (table=TestSplitWalDataLoss:dataloss) id=2 owner=elliott state=RUNNABLE:CREATE_TABLE_PRE_OPERATION added to the store. 2016-01-11 19:21:02,524 DEBUG [ProcedureExecutor-1] lock.ZKInterProcessLockBase(226): Acquired a lock for /hbase/table-lock/TestSplitWalDataLoss:dataloss/write-master:586950000000000 2016-01-11 19:21:02,529 DEBUG [ProcedureExecutor-1] zookeeper.ZKUtil(627): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Unable to get data of znode /hbase/table/TestSplitWalDataLoss:dataloss because node does not exist (not an error) 2016-01-11 19:21:02,532 DEBUG [B.defaultRpcServer.handler=1,queue=0,port=58695] master.MasterRpcServices(1027): Checking to see if procedure is done procId=2 2016-01-11 19:21:02,627 DEBUG [ProcedureExecutor-0] lock.ZKInterProcessLockBase(328): Released /hbase/table-lock/hbase:namespace/write-master:586950000000000 2016-01-11 19:21:02,628 DEBUG [ProcedureExecutor-0] procedure2.ProcedureExecutor(857): Procedure completed in 831msec: CreateTableProcedure (table=hbase:namespace) id=1 owner=elliott state=FINISHED 2016-01-11 19:21:02,636 DEBUG [B.defaultRpcServer.handler=3,queue=0,port=58695] master.MasterRpcServices(1027): Checking to see if procedure is done procId=2 2016-01-11 19:21:02,643 INFO [IPC Server handler 9 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58687 is added to blk_1073741835_1011{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6ab76dd9-6c4e-41f4-9b47-7b827e25bec7:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-70606f4d-e6bd-4292-9765-171d84a1305c:NORMAL|RBW]]} size 0 2016-01-11 19:21:02,644 INFO [IPC Server handler 8 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58684 is added to blk_1073741835_1011{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6ab76dd9-6c4e-41f4-9b47-7b827e25bec7:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-70606f4d-e6bd-4292-9765-171d84a1305c:NORMAL|RBW]]} size 0 2016-01-11 19:21:02,646 DEBUG [ProcedureExecutor-1] util.FSTableDescriptors(656): Wrote descriptor into: hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/.tmp/data/TestSplitWalDataLoss/dataloss/.tabledesc/.tableinfo.0000000001 2016-01-11 19:21:02,647 INFO [RegionOpenAndInitThread-TestSplitWalDataLoss:dataloss-1] regionserver.HRegion(6219): creating HRegion TestSplitWalDataLoss:dataloss HTD == 'TestSplitWalDataLoss:dataloss', {NAME => 'f', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} RootDir = hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/.tmp Table name == TestSplitWalDataLoss:dataloss 2016-01-11 19:21:02,664 INFO [IPC Server handler 0 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58687 is added to blk_1073741836_1012{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-84b79f3b-7553-476e-b313-9382dc0bd9af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-222d7346-39ee-452c-9ca0-e0cb04e5fd92:NORMAL|RBW]]} size 0 2016-01-11 19:21:02,665 INFO [IPC Server handler 1 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58684 is added to blk_1073741836_1012{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-84b79f3b-7553-476e-b313-9382dc0bd9af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-222d7346-39ee-452c-9ca0-e0cb04e5fd92:NORMAL|RBW]]} size 0 2016-01-11 19:21:02,666 DEBUG [RegionOpenAndInitThread-TestSplitWalDataLoss:dataloss-1] regionserver.HRegion(733): Instantiated TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. 2016-01-11 19:21:02,666 DEBUG [RegionOpenAndInitThread-TestSplitWalDataLoss:dataloss-1] regionserver.HRegion(1389): Closing TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06.: disabling compactions & flushes 2016-01-11 19:21:02,666 DEBUG [RegionOpenAndInitThread-TestSplitWalDataLoss:dataloss-1] regionserver.HRegion(1416): Updates disabled for region TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. 2016-01-11 19:21:02,666 INFO [RegionOpenAndInitThread-TestSplitWalDataLoss:dataloss-1] regionserver.HRegion(1519): Closed TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. 2016-01-11 19:21:02,774 INFO [ProcedureExecutor-1] hbase.MetaTableAccessor(1199): Added 1 2016-01-11 19:21:02,839 DEBUG [B.defaultRpcServer.handler=4,queue=0,port=58695] master.MasterRpcServices(1027): Checking to see if procedure is done procId=2 2016-01-11 19:21:02,881 INFO [ProcedureExecutor-1] zookeeper.ZKTableStateManager(100): Moving table TestSplitWalDataLoss:dataloss state from null to ENABLING 2016-01-11 19:21:02,884 DEBUG [ProcedureExecutor-1] balancer.RegionLocationFinder(269): HDFSBlocksDistribution not found in cache for region TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. 2016-01-11 19:21:02,888 DEBUG [ProcedureExecutor-1] regionserver.HRegionFileSystem(201): No StoreFiles for: hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/TestSplitWalDataLoss/dataloss/9e0a9f0464fd954f173089123f349f06/f 2016-01-11 19:21:02,888 INFO [ProcedureExecutor-1] master.AssignmentManager(1613): Assigning 1 region(s) to localhost,58697,1452568859983 2016-01-11 19:21:02,888 DEBUG [ProcedureExecutor-1] zookeeper.ZKAssign(174): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Async create of unassigned node 9e0a9f0464fd954f173089123f349f06 with OFFLINE state 2016-01-11 19:21:02,889 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/region-in-transition 2016-01-11 19:21:02,889 DEBUG [main-EventThread] master.OfflineCallback(69): rs={9e0a9f0464fd954f173089123f349f06 state=OFFLINE, ts=1452568862884, server=null}, server=localhost,58697,1452568859983 2016-01-11 19:21:02,890 DEBUG [main-EventThread] master.OfflineCallback$ExistCallback(106): rs={9e0a9f0464fd954f173089123f349f06 state=OFFLINE, ts=1452568862884, server=null}, server=localhost,58697,1452568859983 2016-01-11 19:21:02,894 DEBUG [ProcedureExecutor-1] master.AssignmentManager(1666): localhost,58697,1452568859983 unassigned znodes=1 of total=1; oldCounter=0 2016-01-11 19:21:02,894 INFO [ProcedureExecutor-1] master.RegionStates(1109): Transition {9e0a9f0464fd954f173089123f349f06 state=OFFLINE, ts=1452568862888, server=null} to {9e0a9f0464fd954f173089123f349f06 state=PENDING_OPEN, ts=1452568862894, server=localhost,58697,1452568859983} 2016-01-11 19:21:02,896 INFO [PriorityRpcServer.handler=3,queue=1,port=58697] regionserver.RSRpcServices(1557): Open TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. 2016-01-11 19:21:02,900 DEBUG [RS_OPEN_REGION-localhost:58697-1] zookeeper.ZKAssign(805): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Transitioning 9e0a9f0464fd954f173089123f349f06 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING 2016-01-11 19:21:02,901 DEBUG [RS_OPEN_REGION-localhost:58697-1] zookeeper.ZKAssign(880): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Transitioned node 9e0a9f0464fd954f173089123f349f06 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING 2016-01-11 19:21:02,901 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/9e0a9f0464fd954f173089123f349f06 2016-01-11 19:21:02,901 DEBUG [RS_OPEN_REGION-localhost:58697-1] regionserver.HRegion(6408): Opening region: {ENCODED => 9e0a9f0464fd954f173089123f349f06, NAME => 'TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06.', STARTKEY => '', ENDKEY => ''} 2016-01-11 19:21:02,902 DEBUG [RS_OPEN_REGION-localhost:58697-1] regionserver.MetricsRegionSourceImpl(70): Creating new MetricsRegionSourceImpl for table dataloss 9e0a9f0464fd954f173089123f349f06 2016-01-11 19:21:02,902 DEBUG [AM.ZK.Worker-pool2-t10] master.AssignmentManager(956): Handling RS_ZK_REGION_OPENING, server=localhost,58697,1452568859983, region=9e0a9f0464fd954f173089123f349f06, current_state={9e0a9f0464fd954f173089123f349f06 state=PENDING_OPEN, ts=1452568862894, server=localhost,58697,1452568859983} 2016-01-11 19:21:02,902 DEBUG [RS_OPEN_REGION-localhost:58697-1] regionserver.HRegion(733): Instantiated TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. 2016-01-11 19:21:02,902 INFO [AM.ZK.Worker-pool2-t10] master.RegionStates(1109): Transition {9e0a9f0464fd954f173089123f349f06 state=PENDING_OPEN, ts=1452568862894, server=localhost,58697,1452568859983} to {9e0a9f0464fd954f173089123f349f06 state=OPENING, ts=1452568862902, server=localhost,58697,1452568859983} 2016-01-11 19:21:02,904 INFO [StoreOpener-9e0a9f0464fd954f173089123f349f06-1] hfile.CacheConfig(292): blockCache=LruBlockCache{blockCount=0, currentSize=1071840, freeSize=1042890464, maxSize=1043962304, heapSize=1071840, minSize=991764160, minFactor=0.95, multiSize=495882080, multiFactor=0.5, singleSize=247941040, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 2016-01-11 19:21:02,904 INFO [StoreOpener-9e0a9f0464fd954f173089123f349f06-1] compactions.CompactionConfiguration(104): size [134217728, 9223372036854775807, 9223372036854775807); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000 2016-01-11 19:21:02,906 DEBUG [StoreOpener-9e0a9f0464fd954f173089123f349f06-1] regionserver.HRegionFileSystem(201): No StoreFiles for: hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/TestSplitWalDataLoss/dataloss/9e0a9f0464fd954f173089123f349f06/f 2016-01-11 19:21:02,907 DEBUG [RS_OPEN_REGION-localhost:58697-1] regionserver.HRegion(3890): Found 0 recovered edits file(s) under hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/TestSplitWalDataLoss/dataloss/9e0a9f0464fd954f173089123f349f06 2016-01-11 19:21:02,909 DEBUG [RS_OPEN_REGION-localhost:58697-1] regionserver.FlushLargeStoresPolicy(56): hbase.hregion.percolumnfamilyflush.size.lower.bound is not specified, use global config(16777216) instead 2016-01-11 19:21:02,912 DEBUG [RS_OPEN_REGION-localhost:58697-1] wal.WALSplitter(729): Wrote region seqId=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/TestSplitWalDataLoss/dataloss/9e0a9f0464fd954f173089123f349f06/recovered.edits/2.seqid to file, newSeqId=2, maxSeqId=0 2016-01-11 19:21:02,912 INFO [RS_OPEN_REGION-localhost:58697-1] regionserver.HRegion(868): Onlined 9e0a9f0464fd954f173089123f349f06; next sequenceid=2 2016-01-11 19:21:02,913 DEBUG [RS_OPEN_REGION-localhost:58697-1] zookeeper.ZKAssign(643): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Attempting to retransition opening state of node 9e0a9f0464fd954f173089123f349f06 2016-01-11 19:21:02,913 INFO [PostOpenDeployTasks:9e0a9f0464fd954f173089123f349f06] regionserver.HRegionServer(1912): Post open deploy tasks for TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. 2016-01-11 19:21:02,915 INFO [PostOpenDeployTasks:9e0a9f0464fd954f173089123f349f06] hbase.MetaTableAccessor(1394): Updated row TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. with server=localhost,58697,1452568859983 2016-01-11 19:21:02,915 DEBUG [PostOpenDeployTasks:9e0a9f0464fd954f173089123f349f06] regionserver.HRegionServer(1946): Finished post open deploy task for TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. 2016-01-11 19:21:02,916 DEBUG [RS_OPEN_REGION-localhost:58697-1] zookeeper.ZKAssign(805): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Transitioning 9e0a9f0464fd954f173089123f349f06 from RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENED 2016-01-11 19:21:02,917 DEBUG [RS_OPEN_REGION-localhost:58697-1] zookeeper.ZKAssign(880): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Transitioned node 9e0a9f0464fd954f173089123f349f06 from RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENED 2016-01-11 19:21:02,917 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/9e0a9f0464fd954f173089123f349f06 2016-01-11 19:21:02,917 DEBUG [RS_OPEN_REGION-localhost:58697-1] coordination.ZkOpenRegionCoordination(91): Transitioned 9e0a9f0464fd954f173089123f349f06 to OPENED in zk on localhost,58697,1452568859983 2016-01-11 19:21:02,917 DEBUG [RS_OPEN_REGION-localhost:58697-1] handler.OpenRegionHandler(170): Opened TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. on localhost,58697,1452568859983 2016-01-11 19:21:02,917 DEBUG [AM.ZK.Worker-pool2-t11] master.AssignmentManager(956): Handling RS_ZK_REGION_OPENED, server=localhost,58697,1452568859983, region=9e0a9f0464fd954f173089123f349f06, current_state={9e0a9f0464fd954f173089123f349f06 state=OPENING, ts=1452568862902, server=localhost,58697,1452568859983} 2016-01-11 19:21:02,917 INFO [AM.ZK.Worker-pool2-t11] master.RegionStates(1109): Transition {9e0a9f0464fd954f173089123f349f06 state=OPENING, ts=1452568862902, server=localhost,58697,1452568859983} to {9e0a9f0464fd954f173089123f349f06 state=OPEN, ts=1452568862917, server=localhost,58697,1452568859983} 2016-01-11 19:21:02,917 DEBUG [AM.ZK.Worker-pool2-t11] coordination.ZkOpenRegionCoordination(350): Handling OPENED of 9e0a9f0464fd954f173089123f349f06 from localhost,58695,1452568859727; deleting unassigned node 2016-01-11 19:21:02,918 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/region-in-transition/9e0a9f0464fd954f173089123f349f06 2016-01-11 19:21:02,918 DEBUG [AM.ZK.Worker-pool2-t11] zookeeper.ZKAssign(479): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Deleted unassigned node 9e0a9f0464fd954f173089123f349f06 in expected state RS_ZK_REGION_OPENED 2016-01-11 19:21:02,918 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/region-in-transition 2016-01-11 19:21:02,918 DEBUG [AM.ZK.Worker-pool2-t12] master.AssignmentManager$4(1345): Znode TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. deleted, state: {9e0a9f0464fd954f173089123f349f06 state=OPEN, ts=1452568862917, server=localhost,58697,1452568859983} 2016-01-11 19:21:02,918 DEBUG [AM.ZK.Worker-pool2-t12] master.RegionStates(456): Onlined 9e0a9f0464fd954f173089123f349f06 on localhost,58697,1452568859983 2016-01-11 19:21:02,918 DEBUG [ProcedureExecutor-1] master.AssignmentManager(1808): Bulk assigning done for localhost,58697,1452568859983 2016-01-11 19:21:02,919 INFO [ProcedureExecutor-1] zookeeper.ZKTableStateManager(100): Moving table TestSplitWalDataLoss:dataloss state from ENABLING to ENABLED 2016-01-11 19:21:03,142 DEBUG [B.defaultRpcServer.handler=2,queue=0,port=58695] master.MasterRpcServices(1027): Checking to see if procedure is done procId=2 2016-01-11 19:21:03,234 DEBUG [ProcedureExecutor-1] lock.ZKInterProcessLockBase(328): Released /hbase/table-lock/TestSplitWalDataLoss:dataloss/write-master:586950000000000 2016-01-11 19:21:03,235 DEBUG [ProcedureExecutor-1] procedure2.ProcedureExecutor(857): Procedure completed in 640msec: CreateTableProcedure (table=TestSplitWalDataLoss:dataloss) id=2 owner=elliott state=FINISHED 2016-01-11 19:21:03,645 DEBUG [B.defaultRpcServer.handler=0,queue=0,port=58695] master.MasterRpcServices(1027): Checking to see if procedure is done procId=2 2016-01-11 19:21:03,646 INFO [main] client.HBaseAdmin$CreateTableFuture(770): Created TestSplitWalDataLoss:dataloss 2016-01-11 19:21:03,649 DEBUG [htable-pool13-t1] ipc.RpcClientImpl$Connection(351): Use SIMPLE authentication for service ClientService, sasl=false 2016-01-11 19:21:03,649 DEBUG [htable-pool13-t1] ipc.RpcClientImpl$Connection(716): Connecting to localhost/127.0.0.1:58697 2016-01-11 19:21:03,650 DEBUG [RpcServer.listener,port=58697] ipc.RpcServer$Listener(879): RpcServer.listener,port=58697: connection from 127.0.0.1:58742; # active connections: 3 2016-01-11 19:21:03,651 INFO [RpcServer.reader=1,bindAddress=localhost,port=58697] ipc.RpcServer$Connection(1712): Auth successful for elliott (auth:SIMPLE) 2016-01-11 19:21:03,651 INFO [RpcServer.reader=1,bindAddress=localhost,port=58697] ipc.RpcServer$Connection(1742): Connection from 127.0.0.1 port: 58742 with version info: version: "1.2.0" url: "git://elliott-mbp/Users/elliott/Code/Public/hbase" revision: "4b59f62c231a39fd4a824c0cbf4c2166c4ec2a76" user: "elliott" date: "Mon Jan 11 19:20:24 PST 2016" src_checksum: "c2d37028b1462df7bac2e95df3bcc070" 2016-01-11 19:21:03,655 INFO [main] zookeeper.RecoverableZooKeeper(120): Process identifier=hconnection-0x4f186450 connecting to ZooKeeper ensemble=localhost:54890 2016-01-11 19:21:03,658 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): hconnection-0x4f1864500x0, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2016-01-11 19:21:03,659 DEBUG [main] ipc.AbstractRpcClient(115): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7fab4be7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2016-01-11 19:21:03,660 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(638): hconnection-0x4f186450-0x15233d9d091000a connected 2016-01-11 19:21:03,661 DEBUG [htable-pool14-t1] ipc.RpcClientImpl$Connection(351): Use SIMPLE authentication for service ClientService, sasl=false 2016-01-11 19:21:03,661 DEBUG [htable-pool14-t1] ipc.RpcClientImpl$Connection(716): Connecting to localhost/127.0.0.1:58697 2016-01-11 19:21:03,661 DEBUG [RpcServer.listener,port=58697] ipc.RpcServer$Listener(879): RpcServer.listener,port=58697: connection from 127.0.0.1:58744; # active connections: 4 2016-01-11 19:21:03,662 INFO [RpcServer.reader=2,bindAddress=localhost,port=58697] ipc.RpcServer$Connection(1712): Auth successful for elliott (auth:SIMPLE) 2016-01-11 19:21:03,662 INFO [RpcServer.reader=2,bindAddress=localhost,port=58697] ipc.RpcServer$Connection(1742): Connection from 127.0.0.1 port: 58744 with version info: version: "1.2.0" url: "git://elliott-mbp/Users/elliott/Code/Public/hbase" revision: "4b59f62c231a39fd4a824c0cbf4c2166c4ec2a76" user: "elliott" date: "Mon Jan 11 19:20:24 PST 2016" src_checksum: "c2d37028b1462df7bac2e95df3bcc070" 2016-01-11 19:21:03,666 INFO [main] hbase.HBaseTestingUtility(2529): getMetaTableRows: row -> TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06.{ENCODED => 9e0a9f0464fd954f173089123f349f06, NAME => 'TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06.', STARTKEY => '', ENDKEY => ''} 2016-01-11 19:21:03,666 INFO [main] client.ConnectionManager$HConnectionImplementation(1712): Closing zookeeper sessionid=0x15233d9d091000a 2016-01-11 19:21:03,667 DEBUG [main] ipc.RpcClientImpl(1147): Stopping rpc client 2016-01-11 19:21:03,667 DEBUG [RpcServer.reader=2,bindAddress=localhost,port=58697] ipc.RpcServer$Listener(911): RpcServer.listener,port=58697: DISCONNECTING client 127.0.0.1:58744 because read count=-1. Number of active connections: 4 2016-01-11 19:21:03,679 DEBUG [main] hbase.HBaseTestingUtility(2556): Found 1 rows for table TestSplitWalDataLoss:dataloss 2016-01-11 19:21:03,679 DEBUG [main] hbase.HBaseTestingUtility(2559): FirstRow=TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. 2016-01-11 19:21:03,903 DEBUG [main] client.ClientSmallReversedScanner(147): Finished with small scan at {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2016-01-11 19:21:03,903 ERROR [main] client.AsyncProcess(404): Failed to get region location org.apache.hadoop.hbase.TableNotFoundException: TestSplitWalDataLoss:dataloss at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1286) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1184) at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:394) at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:343) at org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:238) at org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:190) at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1430) at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1021) at org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss.test(TestSplitWalDataLoss.java:124) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283) at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) 2016-01-11 19:21:03,907 INFO [main] hbase.HBaseTestingUtility(1090): Shutting down minicluster 2016-01-11 19:21:03,907 INFO [main] client.ConnectionManager$HConnectionImplementation(2141): Closing master protocol: MasterService 2016-01-11 19:21:03,907 INFO [main] client.ConnectionManager$HConnectionImplementation(1712): Closing zookeeper sessionid=0x15233d9d0910009 2016-01-11 19:21:03,908 DEBUG [main] ipc.RpcClientImpl(1147): Stopping rpc client 2016-01-11 19:21:03,908 DEBUG [RpcServer.reader=1,bindAddress=localhost,port=58697] ipc.RpcServer$Listener(911): RpcServer.listener,port=58697: DISCONNECTING client 127.0.0.1:58742 because read count=-1. Number of active connections: 3 2016-01-11 19:21:03,908 DEBUG [RpcServer.reader=0,bindAddress=localhost,port=58695] ipc.RpcServer$Listener(911): RpcServer.listener,port=58695: DISCONNECTING client 127.0.0.1:58736 because read count=-1. Number of active connections: 3 2016-01-11 19:21:03,920 DEBUG [main] util.JVMClusterUtil(241): Shutting down HBase Cluster 2016-01-11 19:21:03,920 INFO [main] regionserver.HRegionServer(1878): STOPPED: Cluster shutdown requested 2016-01-11 19:21:03,921 INFO [M:0;localhost:58695] procedure2.ProcedureExecutor(517): Stopping the procedure executor 2016-01-11 19:21:03,921 INFO [M:0;localhost:58695] wal.WALProcedureStore(192): Stopping the WAL Procedure Store 2016-01-11 19:21:03,923 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): regionserver:58702-0x15233d9d0910002, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2016-01-11 19:21:03,923 INFO [main] regionserver.HRegionServer(1878): STOPPED: Shutdown requested 2016-01-11 19:21:03,923 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2016-01-11 19:21:03,923 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2016-01-11 19:21:03,923 DEBUG [main-EventThread] zookeeper.ZKUtil(370): regionserver:58702-0x15233d9d0910002, quorum=localhost:54890, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2016-01-11 19:21:03,923 INFO [RS:0;localhost:58697] regionserver.SplitLogWorker(164): Sending interrupt to stop the worker thread 2016-01-11 19:21:03,923 INFO [main] regionserver.HRegionServer(1878): STOPPED: Shutdown requested 2016-01-11 19:21:03,923 INFO [SplitLogWorker-localhost:58697] regionserver.SplitLogWorker(146): SplitLogWorker interrupted. Exiting. 2016-01-11 19:21:03,923 INFO [RS:0;localhost:58697] regionserver.HeapMemoryManager(202): Stoping HeapMemoryTuner chore. 2016-01-11 19:21:03,923 DEBUG [main-EventThread] zookeeper.ZKUtil(370): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2016-01-11 19:21:03,923 DEBUG [main-EventThread] zookeeper.ZKUtil(370): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2016-01-11 19:21:03,924 INFO [RS:0;localhost:58697] flush.RegionServerFlushTableProcedureManager(115): Stopping region server flush procedure manager gracefully. 2016-01-11 19:21:03,924 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(280): MemStoreFlusher.0 exiting 2016-01-11 19:21:03,924 INFO [MemStoreFlusher.1] regionserver.MemStoreFlusher$FlushHandler(280): MemStoreFlusher.1 exiting 2016-01-11 19:21:03,923 INFO [SplitLogWorker-localhost:58697] regionserver.SplitLogWorker(155): SplitLogWorker localhost,58697,1452568859983 exiting 2016-01-11 19:21:03,923 INFO [RS:1;localhost:58702] regionserver.SplitLogWorker(164): Sending interrupt to stop the worker thread 2016-01-11 19:21:03,924 INFO [RS:0;localhost:58697] snapshot.RegionServerSnapshotManager(136): Stopping RegionServerSnapshotManager gracefully. 2016-01-11 19:21:03,924 INFO [SplitLogWorker-localhost:58702] regionserver.SplitLogWorker(146): SplitLogWorker interrupted. Exiting. 2016-01-11 19:21:03,924 INFO [RS:1;localhost:58702] regionserver.HeapMemoryManager(202): Stoping HeapMemoryTuner chore. 2016-01-11 19:21:03,924 INFO [SplitLogWorker-localhost:58702] regionserver.SplitLogWorker(155): SplitLogWorker localhost,58702,1452568860027 exiting 2016-01-11 19:21:03,925 INFO [RS:1;localhost:58702] flush.RegionServerFlushTableProcedureManager(115): Stopping region server flush procedure manager gracefully. 2016-01-11 19:21:03,925 INFO [RS:1;localhost:58702] snapshot.RegionServerSnapshotManager(136): Stopping RegionServerSnapshotManager gracefully. 2016-01-11 19:21:03,924 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(280): MemStoreFlusher.0 exiting 2016-01-11 19:21:03,925 INFO [RS:1;localhost:58702] regionserver.HRegionServer(1032): stopping server localhost,58702,1452568860027 2016-01-11 19:21:03,925 INFO [MemStoreFlusher.1] regionserver.MemStoreFlusher$FlushHandler(280): MemStoreFlusher.1 exiting 2016-01-11 19:21:03,925 DEBUG [RS:1;localhost:58702] zookeeper.MetaTableLocator(616): Stopping MetaTableLocator 2016-01-11 19:21:03,925 INFO [RS:1;localhost:58702] client.ConnectionManager$HConnectionImplementation(1712): Closing zookeeper sessionid=0x15233d9d0910004 2016-01-11 19:21:03,926 DEBUG [RS:1;localhost:58702] ipc.RpcClientImpl(1147): Stopping rpc client 2016-01-11 19:21:03,926 INFO [RS:1;localhost:58702] regionserver.HRegionServer(1060): stopping server localhost,58702,1452568860027; all regions closed. 2016-01-11 19:21:03,927 DEBUG [RS:1;localhost:58702] wal.FSHLog(1063): Closing WAL writer in /user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/WALs/localhost,58702,1452568860027 2016-01-11 19:21:03,929 DEBUG [RS_CLOSE_REGION-localhost:58697-0] handler.CloseRegionHandler(119): Processing close of TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. 2016-01-11 19:21:03,929 INFO [RS:0;localhost:58697] regionserver.HRegionServer(2809): Received CLOSE for the region: 9e0a9f0464fd954f173089123f349f06, which we are already trying to CLOSE, but not completed yet 2016-01-11 19:21:03,930 DEBUG [RS_CLOSE_REGION-localhost:58697-0] regionserver.HRegion(1389): Closing TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06.: disabling compactions & flushes 2016-01-11 19:21:03,930 WARN [RS:0;localhost:58697] regionserver.HRegionServer(2748): Failed to close TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. - ignoring and continuing org.apache.hadoop.hbase.regionserver.RegionAlreadyInTransitionException: The region 9e0a9f0464fd954f173089123f349f06 was already closing. New CLOSE request is ignored. at org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegion(HRegionServer.java:2817) at org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegionIgnoreErrors(HRegionServer.java:2743) at org.apache.hadoop.hbase.regionserver.HRegionServer.closeUserRegions(HRegionServer.java:2375) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1031) at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:156) at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:108) at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:140) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:334) at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:138) at java.lang.Thread.run(Thread.java:745) 2016-01-11 19:21:03,929 DEBUG [RS_CLOSE_REGION-localhost:58697-1] handler.CloseRegionHandler(119): Processing close of hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. 2016-01-11 19:21:03,930 INFO [RS:0;localhost:58697] regionserver.HRegionServer(1032): stopping server localhost,58697,1452568859983 2016-01-11 19:21:03,931 DEBUG [RS:0;localhost:58697] zookeeper.MetaTableLocator(616): Stopping MetaTableLocator 2016-01-11 19:21:03,930 DEBUG [RS_CLOSE_REGION-localhost:58697-0] regionserver.HRegion(1416): Updates disabled for region TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. 2016-01-11 19:21:03,931 DEBUG [RS_CLOSE_REGION-localhost:58697-1] regionserver.HRegion(1389): Closing hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b.: disabling compactions & flushes 2016-01-11 19:21:03,931 INFO [RS:0;localhost:58697] client.ConnectionManager$HConnectionImplementation(1712): Closing zookeeper sessionid=0x15233d9d0910003 2016-01-11 19:21:03,932 INFO [StoreCloserThread-TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06.-1] regionserver.HStore(875): Closed f 2016-01-11 19:21:03,932 DEBUG [RS_CLOSE_REGION-localhost:58697-1] regionserver.HRegion(1416): Updates disabled for region hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. 2016-01-11 19:21:03,932 INFO [IPC Server handler 2 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58684 is added to blk_1073741829_1005{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-70606f4d-e6bd-4292-9765-171d84a1305c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-84b79f3b-7553-476e-b313-9382dc0bd9af:NORMAL|RBW]]} size 83 2016-01-11 19:21:03,933 DEBUG [RS:0;localhost:58697] ipc.RpcClientImpl(1147): Stopping rpc client 2016-01-11 19:21:03,932 INFO [RS_CLOSE_REGION-localhost:58697-1] regionserver.HRegion(2157): Flushing 1/1 column families, memstore=544 B 2016-01-11 19:21:03,933 INFO [RS:0;localhost:58697] regionserver.CompactSplitThread(403): Waiting for Split Thread to finish... 2016-01-11 19:21:03,934 INFO [RS:0;localhost:58697] regionserver.CompactSplitThread(403): Waiting for Merge Thread to finish... 2016-01-11 19:21:03,934 INFO [RS:0;localhost:58697] regionserver.CompactSplitThread(403): Waiting for Large Compaction Thread to finish... 2016-01-11 19:21:03,934 INFO [RS:0;localhost:58697] regionserver.CompactSplitThread(403): Waiting for Small Compaction Thread to finish... 2016-01-11 19:21:03,933 INFO [IPC Server handler 4 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58687 is added to blk_1073741829_1005{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-70606f4d-e6bd-4292-9765-171d84a1305c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-84b79f3b-7553-476e-b313-9382dc0bd9af:NORMAL|RBW]]} size 83 2016-01-11 19:21:03,934 INFO [RS:0;localhost:58697] regionserver.HRegionServer(1261): Waiting on 3 regions to close 2016-01-11 19:21:03,934 DEBUG [RS:0;localhost:58697] regionserver.HRegionServer(1265): {1588230740=TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06., 33d6151ed04c075d7291124431dccb8b=hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b., 9e0a9f0464fd954f173089123f349f06=TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06.} 2016-01-11 19:21:03,938 INFO [IPC Server handler 0 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58684 is added to blk_1073741832_1008{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-222d7346-39ee-452c-9ca0-e0cb04e5fd92:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-84b79f3b-7553-476e-b313-9382dc0bd9af:NORMAL|RBW]]} size 473 2016-01-11 19:21:03,938 INFO [IPC Server handler 3 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58687 is added to blk_1073741832_1008{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-222d7346-39ee-452c-9ca0-e0cb04e5fd92:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-84b79f3b-7553-476e-b313-9382dc0bd9af:NORMAL|RBW]]} size 473 2016-01-11 19:21:03,939 INFO [M:0;localhost:58695] regionserver.HRegionServer(1032): stopping server localhost,58695,1452568859727 2016-01-11 19:21:03,939 DEBUG [M:0;localhost:58695] zookeeper.MetaTableLocator(616): Stopping MetaTableLocator 2016-01-11 19:21:03,942 INFO [M:0;localhost:58695] client.ConnectionManager$HConnectionImplementation(1712): Closing zookeeper sessionid=0x15233d9d0910005 2016-01-11 19:21:03,944 DEBUG [RS_CLOSE_REGION-localhost:58697-0] wal.WALSplitter(729): Wrote region seqId=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/TestSplitWalDataLoss/dataloss/9e0a9f0464fd954f173089123f349f06/recovered.edits/4.seqid to file, newSeqId=4, maxSeqId=2 2016-01-11 19:21:03,944 DEBUG [M:0;localhost:58695] ipc.RpcClientImpl(1147): Stopping rpc client 2016-01-11 19:21:03,944 DEBUG [RS:1;localhost:58702] wal.FSHLog(1021): Moved 1 WAL file(s) to /user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/oldWALs 2016-01-11 19:21:03,944 DEBUG [RpcServer.reader=2,bindAddress=localhost,port=58697] ipc.RpcServer$Listener(911): RpcServer.listener,port=58697: DISCONNECTING client 127.0.0.1:58725 because read count=-1. Number of active connections: 2 2016-01-11 19:21:03,944 INFO [RS:1;localhost:58702] wal.FSHLog(1024): Closed WAL: FSHLog localhost%2C58702%2C1452568860027.default:(num 1452568860561) 2016-01-11 19:21:03,945 DEBUG [RS:1;localhost:58702] ipc.RpcClientImpl(1147): Stopping rpc client 2016-01-11 19:21:03,946 DEBUG [RpcServer.reader=1,bindAddress=localhost,port=58695] ipc.RpcServer$Listener(911): RpcServer.listener,port=58695: DISCONNECTING client 127.0.0.1:58713 because read count=-1. Number of active connections: 2 2016-01-11 19:21:03,947 INFO [RS_CLOSE_REGION-localhost:58697-0] regionserver.HRegion(1519): Closed TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. 2016-01-11 19:21:03,947 DEBUG [RS_CLOSE_REGION-localhost:58697-0] handler.CloseRegionHandler(164): Closed TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. 2016-01-11 19:21:03,954 INFO [M:0;localhost:58695] regionserver.HRegionServer(1060): stopping server localhost,58695,1452568859727; all regions closed. 2016-01-11 19:21:03,954 INFO [M:0;localhost:58695] hbase.ChoreService(323): Chore service for: localhost,58695,1452568859727 had [[ScheduledChore: Name: localhost,58695,1452568859727-RegionNormalizerChore Period: 1800000 Unit: MILLISECONDS], [ScheduledChore: Name: localhost,58695,1452568859727-ClusterStatusChore Period: 60000 Unit: MILLISECONDS], [ScheduledChore: Name: CatalogJanitor-localhost:58695 Period: 300000 Unit: MILLISECONDS], [ScheduledChore: Name: HFileCleaner Period: 60000 Unit: MILLISECONDS], [ScheduledChore: Name: LogsCleaner Period: 60000 Unit: MILLISECONDS], [ScheduledChore: Name: localhost,58695,1452568859727-BalancerChore Period: 300000 Unit: MILLISECONDS]] on shutdown 2016-01-11 19:21:03,955 INFO [M:0;localhost:58695] master.ServerManager(557): Waiting on regionserver(s) to go down localhost,58697,1452568859983, localhost,58702,1452568860027 2016-01-11 19:21:03,955 INFO [RS:1;localhost:58702] regionserver.Leases(146): RS:1;localhost:58702 closing leases 2016-01-11 19:21:03,956 INFO [RS:1;localhost:58702] regionserver.Leases(149): RS:1;localhost:58702 closed leases 2016-01-11 19:21:03,956 INFO [RS:1;localhost:58702] hbase.ChoreService(323): Chore service for: localhost,58702,1452568860027 had [[ScheduledChore: Name: localhost,58702,1452568860027-MemstoreFlusherChore Period: 1000 Unit: MILLISECONDS], [ScheduledChore: Name: MovedRegionsCleaner for region localhost,58702,1452568860027 Period: 120000 Unit: MILLISECONDS]] on shutdown 2016-01-11 19:21:04,004 INFO [IPC Server handler 5 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58684 is added to blk_1073741837_1013{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-70606f4d-e6bd-4292-9765-171d84a1305c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-6ab76dd9-6c4e-41f4-9b47-7b827e25bec7:NORMAL|RBW]]} size 0 2016-01-11 19:21:04,005 INFO [IPC Server handler 6 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58687 is added to blk_1073741837_1013{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-70606f4d-e6bd-4292-9765-171d84a1305c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-6ab76dd9-6c4e-41f4-9b47-7b827e25bec7:NORMAL|RBW]]} size 0 2016-01-11 19:21:04,005 INFO [RS_CLOSE_REGION-localhost:58697-1] regionserver.DefaultStoreFlusher(90): Flushed, sequenceid=8, memsize=544, hasBloomFilter=true, into tmp file hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/namespace/33d6151ed04c075d7291124431dccb8b/.tmp/bdbf75a2e3b9401cb48487398573c710 2016-01-11 19:21:04,053 DEBUG [RS_CLOSE_REGION-localhost:58697-1] regionserver.HRegionFileSystem(381): Committing store file hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/namespace/33d6151ed04c075d7291124431dccb8b/.tmp/bdbf75a2e3b9401cb48487398573c710 as hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/namespace/33d6151ed04c075d7291124431dccb8b/info/bdbf75a2e3b9401cb48487398573c710 2016-01-11 19:21:04,060 INFO [RS_CLOSE_REGION-localhost:58697-1] regionserver.HStore(968): Added hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/namespace/33d6151ed04c075d7291124431dccb8b/info/bdbf75a2e3b9401cb48487398573c710, entries=3, sequenceid=8, filesize=4.9 K 2016-01-11 19:21:04,062 INFO [RS_CLOSE_REGION-localhost:58697-1] regionserver.HRegion(2454): Finished memstore flush of ~544 B/544, currentsize=0 B/0 for region hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. in 130ms, sequenceid=8, compaction requested=false 2016-01-11 19:21:04,064 INFO [StoreCloserThread-hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b.-1] regionserver.HStore(875): Closed info 2016-01-11 19:21:04,070 DEBUG [RS_CLOSE_REGION-localhost:58697-1] wal.WALSplitter(729): Wrote region seqId=hdfs://localhost:58683/user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/data/hbase/namespace/33d6151ed04c075d7291124431dccb8b/recovered.edits/11.seqid to file, newSeqId=11, maxSeqId=2 2016-01-11 19:21:04,071 INFO [RS_CLOSE_REGION-localhost:58697-1] regionserver.HRegion(1519): Closed hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. 2016-01-11 19:21:04,071 DEBUG [RS_CLOSE_REGION-localhost:58697-1] handler.CloseRegionHandler(164): Closed hbase:namespace,,1452568861684.33d6151ed04c075d7291124431dccb8b. 2016-01-11 19:21:04,135 ERROR [RS:0;localhost:58697] regionserver.HRegionServer(2822): Received CLOSE for a region which is not online, and we're not opening. 2016-01-11 19:21:04,135 WARN [RS:0;localhost:58697] regionserver.HRegionServer(2748): Failed to close TestSplitWalDataLoss:dataloss,,1452568862485.9e0a9f0464fd954f173089123f349f06. - ignoring and continuing org.apache.hadoop.hbase.NotServingRegionException: The region 9e0a9f0464fd954f173089123f349f06 is not online, and is not opening. at org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegion(HRegionServer.java:2825) at org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegionIgnoreErrors(HRegionServer.java:2743) at org.apache.hadoop.hbase.regionserver.HRegionServer.waitOnAllRegionsToClose(HRegionServer.java:1278) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1059) at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:156) at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:108) at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:140) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:334) at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:138) at java.lang.Thread.run(Thread.java:745) 2016-01-11 19:21:04,135 INFO [RS:0;localhost:58697] regionserver.HRegionServer(1284): We were exiting though online regions are not empty, because some regions failed closing 2016-01-11 19:21:04,135 INFO [RS:0;localhost:58697] regionserver.HRegionServer(1060): stopping server localhost,58697,1452568859983; all regions closed. 2016-01-11 19:21:04,135 DEBUG [RS:0;localhost:58697] wal.FSHLog(1063): Closing WAL writer in /user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/WALs/localhost,58697,1452568859983 2016-01-11 19:21:04,138 INFO [IPC Server handler 8 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58687 is added to blk_1073741831_1007{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-70606f4d-e6bd-4292-9765-171d84a1305c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-6ab76dd9-6c4e-41f4-9b47-7b827e25bec7:NORMAL|RBW]]} size 83 2016-01-11 19:21:04,139 INFO [IPC Server handler 7 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58684 is added to blk_1073741831_1007{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-70606f4d-e6bd-4292-9765-171d84a1305c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-6ab76dd9-6c4e-41f4-9b47-7b827e25bec7:NORMAL|RBW]]} size 83 2016-01-11 19:21:04,141 DEBUG [RS:0;localhost:58697] wal.FSHLog(1021): Moved 1 WAL file(s) to /user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/oldWALs 2016-01-11 19:21:04,142 INFO [RS:0;localhost:58697] wal.FSHLog(1024): Closed WAL: FSHLog localhost%2C58697%2C1452568859983..meta:.meta(num 1452568861410) 2016-01-11 19:21:04,142 DEBUG [RS:0;localhost:58697] wal.FSHLog(1063): Closing WAL writer in /user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/WALs/localhost,58697,1452568859983 2016-01-11 19:21:04,144 INFO [IPC Server handler 4 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58687 is added to blk_1073741830_1006{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6ab76dd9-6c4e-41f4-9b47-7b827e25bec7:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-222d7346-39ee-452c-9ca0-e0cb04e5fd92:NORMAL|RBW]]} size 83 2016-01-11 19:21:04,144 INFO [IPC Server handler 5 on 58683] blockmanagement.BlockManager(2383): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58684 is added to blk_1073741830_1006{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-6ab76dd9-6c4e-41f4-9b47-7b827e25bec7:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-222d7346-39ee-452c-9ca0-e0cb04e5fd92:NORMAL|RBW]]} size 83 2016-01-11 19:21:04,147 DEBUG [RS:0;localhost:58697] wal.FSHLog(1021): Moved 1 WAL file(s) to /user/elliott/test-data/cc29befd-a4c1-41fb-8a8c-fc9f0ac481bb/oldWALs 2016-01-11 19:21:04,147 INFO [RS:0;localhost:58697] wal.FSHLog(1024): Closed WAL: FSHLog localhost%2C58697%2C1452568859983.default:(num 1452568860561) 2016-01-11 19:21:04,147 DEBUG [RS:0;localhost:58697] ipc.RpcClientImpl(1147): Stopping rpc client 2016-01-11 19:21:04,147 DEBUG [RpcServer.reader=2,bindAddress=localhost,port=58695] ipc.RpcServer$Listener(911): RpcServer.listener,port=58695: DISCONNECTING client 127.0.0.1:58714 because read count=-1. Number of active connections: 1 2016-01-11 19:21:04,157 INFO [RS:0;localhost:58697] regionserver.Leases(146): RS:0;localhost:58697 closing leases 2016-01-11 19:21:04,157 INFO [RS:0;localhost:58697] regionserver.Leases(149): RS:0;localhost:58697 closed leases 2016-01-11 19:21:04,157 INFO [RS:0;localhost:58697] hbase.ChoreService(323): Chore service for: localhost,58697,1452568859983 had [[ScheduledChore: Name: localhost,58697,1452568859983-MemstoreFlusherChore Period: 1000 Unit: MILLISECONDS], [ScheduledChore: Name: MovedRegionsCleaner for region localhost,58697,1452568859983 Period: 120000 Unit: MILLISECONDS]] on shutdown 2016-01-11 19:21:04,265 INFO [localhost,58695,1452568859727_splitLogManager__ChoreService_1] hbase.ScheduledChore(179): Chore: SplitLogManager Timeout Monitor was stopped 2016-01-11 19:21:04,412 INFO [RS_OPEN_META-localhost:58697-0-MetaLogRoller] regionserver.LogRoller(176): LogRoller exiting. 2016-01-11 19:21:04,656 INFO [regionserver/localhost/127.0.0.1:0.logRoller] regionserver.LogRoller(176): LogRoller exiting. 2016-01-11 19:21:04,656 INFO [regionserver/localhost/127.0.0.1:0.leaseChecker] regionserver.Leases(146): regionserver/localhost/127.0.0.1:0.leaseChecker closing leases 2016-01-11 19:21:04,656 INFO [regionserver/localhost/127.0.0.1:0.leaseChecker] regionserver.Leases(149): regionserver/localhost/127.0.0.1:0.leaseChecker closed leases 2016-01-11 19:21:04,656 INFO [RS:1;localhost:58702] regionserver.CompactSplitThread(403): Waiting for Split Thread to finish... 2016-01-11 19:21:04,657 INFO [RS:1;localhost:58702] regionserver.CompactSplitThread(403): Waiting for Merge Thread to finish... 2016-01-11 19:21:04,657 INFO [RS:1;localhost:58702] regionserver.CompactSplitThread(403): Waiting for Large Compaction Thread to finish... 2016-01-11 19:21:04,657 INFO [RS:1;localhost:58702] regionserver.CompactSplitThread(403): Waiting for Small Compaction Thread to finish... 2016-01-11 19:21:04,656 INFO [regionserver/localhost/127.0.0.1:0.logRoller] regionserver.LogRoller(176): LogRoller exiting. 2016-01-11 19:21:04,656 INFO [regionserver/localhost/127.0.0.1:0.leaseChecker] regionserver.Leases(146): regionserver/localhost/127.0.0.1:0.leaseChecker closing leases 2016-01-11 19:21:04,657 INFO [regionserver/localhost/127.0.0.1:0.leaseChecker] regionserver.Leases(149): regionserver/localhost/127.0.0.1:0.leaseChecker closed leases 2016-01-11 19:21:04,662 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/replication/rs/localhost,58697,1452568859983 2016-01-11 19:21:04,662 INFO [RS:0;localhost:58697] ipc.RpcServer(2318): Stopping server on 58697 2016-01-11 19:21:04,662 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): regionserver:58702-0x15233d9d0910002, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/replication/rs/localhost,58702,1452568860027 2016-01-11 19:21:04,662 INFO [RpcServer.listener,port=58697] ipc.RpcServer$Listener(815): RpcServer.listener,port=58697: stopping 2016-01-11 19:21:04,663 INFO [RS:1;localhost:58702] ipc.RpcServer(2318): Stopping server on 58702 2016-01-11 19:21:04,663 INFO [RpcServer.listener,port=58702] ipc.RpcServer$Listener(815): RpcServer.listener,port=58702: stopping 2016-01-11 19:21:04,664 INFO [RpcServer.responder] ipc.RpcServer$Responder(1057): RpcServer.responder: stopped 2016-01-11 19:21:04,664 INFO [RpcServer.responder] ipc.RpcServer$Responder(960): RpcServer.responder: stopping 2016-01-11 19:21:04,664 INFO [RpcServer.responder] ipc.RpcServer$Responder(1057): RpcServer.responder: stopped 2016-01-11 19:21:04,664 INFO [RpcServer.responder] ipc.RpcServer$Responder(960): RpcServer.responder: stopping 2016-01-11 19:21:04,665 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): regionserver:58702-0x15233d9d0910002, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/localhost,58697,1452568859983 2016-01-11 19:21:04,665 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/localhost,58697,1452568859983 2016-01-11 19:21:04,665 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): regionserver:58702-0x15233d9d0910002, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2016-01-11 19:21:04,665 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/localhost,58697,1452568859983 2016-01-11 19:21:04,666 INFO [main-EventThread] zookeeper.RegionServerTracker(118): RegionServer ephemeral node deleted, processing expiration [localhost,58697,1452568859983] 2016-01-11 19:21:04,665 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2016-01-11 19:21:04,666 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): regionserver:58702-0x15233d9d0910002, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/localhost,58702,1452568860027 2016-01-11 19:21:04,666 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): regionserver:58697-0x15233d9d0910001, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/localhost,58702,1452568860027 2016-01-11 19:21:04,666 INFO [main-EventThread] master.ServerManager(612): Cluster shutdown set; localhost,58697,1452568859983 expired; onlineServers=1 2016-01-11 19:21:04,666 INFO [RS:0;localhost:58697] regionserver.HRegionServer(1104): stopping server localhost,58697,1452568859983; zookeeper connection closed. 2016-01-11 19:21:04,667 INFO [RS:1;localhost:58702] regionserver.HRegionServer(1104): stopping server localhost,58702,1452568860027; zookeeper connection closed. 2016-01-11 19:21:04,667 INFO [RS:1;localhost:58702] regionserver.HRegionServer(1107): RS:1;localhost:58702 exiting 2016-01-11 19:21:04,667 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2016-01-11 19:21:04,667 INFO [RS:0;localhost:58697] regionserver.HRegionServer(1107): RS:0;localhost:58697 exiting 2016-01-11 19:21:04,667 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/localhost,58702,1452568860027 2016-01-11 19:21:04,667 INFO [main-EventThread] zookeeper.RegionServerTracker(118): RegionServer ephemeral node deleted, processing expiration [localhost,58702,1452568860027] 2016-01-11 19:21:04,667 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@415279c7] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(191): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@415279c7 2016-01-11 19:21:04,667 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@30249098] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(191): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@30249098 2016-01-11 19:21:04,667 INFO [main-EventThread] master.ServerManager(612): Cluster shutdown set; localhost,58702,1452568860027 expired; onlineServers=0 2016-01-11 19:21:04,668 INFO [M:0;localhost:58695] master.ServerManager(565): ZK shows there is only the master self online, exiting now 2016-01-11 19:21:04,668 DEBUG [M:0;localhost:58695] master.HMaster(1125): Stopping service threads 2016-01-11 19:21:04,668 INFO [main] util.JVMClusterUtil(317): Shutdown of 1 master(s) and 2 regionserver(s) complete 2016-01-11 19:21:04,669 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher(554): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2016-01-11 19:21:04,669 INFO [M:0;localhost:58695] client.ConnectionManager$HConnectionImplementation(1712): Closing zookeeper sessionid=0x15233d9d0910006 2016-01-11 19:21:04,669 DEBUG [main-EventThread] zookeeper.ZKUtil(370): master:58695-0x15233d9d0910000, quorum=localhost:54890, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2016-01-11 19:21:04,669 DEBUG [M:0;localhost:58695] ipc.RpcClientImpl(1147): Stopping rpc client 2016-01-11 19:21:04,683 INFO [M:0;localhost:58695] hbase.ChoreService(323): Chore service for: localhost,58695,1452568859727_splitLogManager_ had [] on shutdown 2016-01-11 19:21:04,683 INFO [M:0;localhost:58695] flush.MasterFlushTableProcedureManager(79): stop: server shutting down. 2016-01-11 19:21:04,683 INFO [M:0;localhost:58695] ipc.RpcServer(2318): Stopping server on 58695 2016-01-11 19:21:04,683 INFO [RpcServer.listener,port=58695] ipc.RpcServer$Listener(815): RpcServer.listener,port=58695: stopping 2016-01-11 19:21:04,684 INFO [RpcServer.responder] ipc.RpcServer$Responder(1057): RpcServer.responder: stopped 2016-01-11 19:21:04,684 INFO [RpcServer.responder] ipc.RpcServer$Responder(960): RpcServer.responder: stopping 2016-01-11 19:21:04,685 DEBUG [M:0;localhost:58695] zookeeper.RecoverableZooKeeper(188): Node /hbase/rs/localhost,58695,1452568859727 already deleted, retry=false 2016-01-11 19:21:04,686 INFO [M:0;localhost:58695] regionserver.HRegionServer(1104): stopping server localhost,58695,1452568859727; zookeeper connection closed. 2016-01-11 19:21:04,686 INFO [M:0;localhost:58695] regionserver.HRegionServer(1107): M:0;localhost:58695 exiting 2016-01-11 19:21:04,687 INFO [main] zookeeper.MiniZooKeeperCluster(316): Shutdown MiniZK cluster with all ZK servers 2016-01-11 19:21:04,688 WARN [main] datanode.DirectoryScanner(375): DirectoryScanner: shutdown has been called 2016-01-11 19:21:04,691 INFO [main] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2016-01-11 19:21:04,787 DEBUG [localhost:58695.activeMasterManager-EventThread] zookeeper.ZooKeeperWatcher(554): replicationLogCleaner-0x15233d9d0910007, quorum=localhost:54890, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Disconnected, path=null 2016-01-11 19:21:04,787 DEBUG [localhost:58695.activeMasterManager-EventThread] zookeeper.ZooKeeperWatcher(643): replicationLogCleaner-0x15233d9d0910007, quorum=localhost:54890, baseZNode=/hbase Received Disconnected from ZooKeeper, ignoring 2016-01-11 19:21:04,798 WARN [DataNode: [[[DISK]file:/Users/elliott/Code/Public/hbase/hbase-server/target/test-data/e0ff3de9-2b27-45bb-ae9b-800547bf5540/dfscluster_d09e3e7b-f697-4440-a7e3-d570985bf1a3/dfs/data/data3/, [DISK]file:/Users/elliott/Code/Public/hbase/hbase-server/target/test-data/e0ff3de9-2b27-45bb-ae9b-800547bf5540/dfscluster_d09e3e7b-f697-4440-a7e3-d570985bf1a3/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:58683] datanode.BPServiceActor(722): BPOfferService for Block pool BP-969595829-127.0.0.1-1452568857091 (Datanode Uuid 39ef380c-c889-4f94-ac05-c6b51ea2ad87) service to localhost/127.0.0.1:58683 interrupted 2016-01-11 19:21:04,798 WARN [DataNode: [[[DISK]file:/Users/elliott/Code/Public/hbase/hbase-server/target/test-data/e0ff3de9-2b27-45bb-ae9b-800547bf5540/dfscluster_d09e3e7b-f697-4440-a7e3-d570985bf1a3/dfs/data/data3/, [DISK]file:/Users/elliott/Code/Public/hbase/hbase-server/target/test-data/e0ff3de9-2b27-45bb-ae9b-800547bf5540/dfscluster_d09e3e7b-f697-4440-a7e3-d570985bf1a3/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:58683] datanode.BPServiceActor(845): Ending block pool service for: Block pool BP-969595829-127.0.0.1-1452568857091 (Datanode Uuid 39ef380c-c889-4f94-ac05-c6b51ea2ad87) service to localhost/127.0.0.1:58683 2016-01-11 19:21:04,800 WARN [main] datanode.DirectoryScanner(375): DirectoryScanner: shutdown has been called 2016-01-11 19:21:04,803 INFO [main] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2016-01-11 19:21:04,910 WARN [DataNode: [[[DISK]file:/Users/elliott/Code/Public/hbase/hbase-server/target/test-data/e0ff3de9-2b27-45bb-ae9b-800547bf5540/dfscluster_d09e3e7b-f697-4440-a7e3-d570985bf1a3/dfs/data/data1/, [DISK]file:/Users/elliott/Code/Public/hbase/hbase-server/target/test-data/e0ff3de9-2b27-45bb-ae9b-800547bf5540/dfscluster_d09e3e7b-f697-4440-a7e3-d570985bf1a3/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:58683] datanode.BPServiceActor(722): BPOfferService for Block pool BP-969595829-127.0.0.1-1452568857091 (Datanode Uuid 1c4e8d3e-68ce-4377-bddf-c64ec7768a10) service to localhost/127.0.0.1:58683 interrupted 2016-01-11 19:21:04,910 WARN [DataNode: [[[DISK]file:/Users/elliott/Code/Public/hbase/hbase-server/target/test-data/e0ff3de9-2b27-45bb-ae9b-800547bf5540/dfscluster_d09e3e7b-f697-4440-a7e3-d570985bf1a3/dfs/data/data1/, [DISK]file:/Users/elliott/Code/Public/hbase/hbase-server/target/test-data/e0ff3de9-2b27-45bb-ae9b-800547bf5540/dfscluster_d09e3e7b-f697-4440-a7e3-d570985bf1a3/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:58683] datanode.BPServiceActor(845): Ending block pool service for: Block pool BP-969595829-127.0.0.1-1452568857091 (Datanode Uuid 1c4e8d3e-68ce-4377-bddf-c64ec7768a10) service to localhost/127.0.0.1:58683 2016-01-11 19:21:04,918 WARN [org.apache.hadoop.hdfs.server.blockmanagement.DecommissionManager$Monitor@2ece4966] blockmanagement.DecommissionManager$Monitor(78): Monitor interrupted: java.lang.InterruptedException: sleep interrupted 2016-01-11 19:21:04,929 INFO [main] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2016-01-11 19:21:05,066 INFO [main] hbase.HBaseTestingUtility(1103): Minicluster is down 2016-01-11 19:21:05,102 INFO [main] hbase.ResourceChecker(171): after: regionserver.TestSplitWalDataLoss#test Thread=26 (was 4) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) Potentially hanging thread: IPC Client (852303090) connection to localhost/127.0.0.1:58683 from elliott java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:904) org.apache.hadoop.ipc.Client$Connection.run(Client.java:949) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) Potentially hanging thread: process reaper sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) Potentially hanging thread: Timer for 'NameNode' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: IPC Client (852303090) connection to localhost/127.0.0.1:58683 from elliott.hfs.1 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:904) org.apache.hadoop.ipc.Client$Connection.run(Client.java:949) Potentially hanging thread: LeaseRenewer:elliott@localhost:58683 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:438) org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71) org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:298) java.lang.Thread.run(Thread.java:745) Potentially hanging thread: process reaper sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) Potentially hanging thread: SessionTracker java.lang.Object.wait(Native Method) org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146) Potentially hanging thread: process reaper sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) Potentially hanging thread: LeaseRenewer:elliott.hfs.1@localhost:58683 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:438) org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71) org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:298) java.lang.Thread.run(Thread.java:745) Potentially hanging thread: IPC Client (852303090) connection to localhost/127.0.0.1:58683 from elliott java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:904) org.apache.hadoop.ipc.Client$Connection.run(Client.java:949) Potentially hanging thread: region-location-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) Potentially hanging thread: localhost:58695.activeMasterManager-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) Potentially hanging thread: IPC Parameter Sending Thread #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) Potentially hanging thread: process reaper sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) Potentially hanging thread: snapshot-log-cleaner-cache-refresher java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: snapshot-hfile-cleaner-cache-refresher java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: org.apache.hadoop.hdfs.PeerCache@78520761 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:244) org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:41) org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:119) java.lang.Thread.run(Thread.java:745) Potentially hanging thread: LeaseRenewer:elliott.hfs.0@localhost:58683 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:438) org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71) org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:298) java.lang.Thread.run(Thread.java:745) Potentially hanging thread: localhost:58695.activeMasterManager-SendThread(localhost:54890) java.lang.Thread.sleep(Native Method) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994) Potentially hanging thread: IPC Client (852303090) connection to localhost/127.0.0.1:58683 from elliott.hfs.0 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:904) org.apache.hadoop.ipc.Client$Connection.run(Client.java:949) - Thread LEAK? -, OpenFileDescriptor=253 (was 168) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=10240 (was 10240), SystemLoadAverage=1144 (was 985) - SystemLoadAverage LEAK? -, ProcessCount=288 (was 288), AvailableMemoryMB=49 (was 133), ConnectionCount=0 (was 0) 2016-01-11 19:21:05,105 INFO [Thread-1] regionserver.ShutdownHook$ShutdownHookThread(111): Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@7ff95560 2016-01-11 19:21:05,105 INFO [Thread-1] regionserver.ShutdownHook$ShutdownHookThread(133): Shutdown hook finished. 2016-01-11 19:21:05,105 INFO [Thread-1] regionserver.ShutdownHook$ShutdownHookThread(111): Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@7ff95560 2016-01-11 19:21:05,105 INFO [Thread-1] regionserver.ShutdownHook$ShutdownHookThread(120): Starting fs shutdown hook thread. 2016-01-11 19:21:05,105 INFO [Thread-1] regionserver.ShutdownHook$ShutdownHookThread(133): Shutdown hook finished.