Stacktrace java.lang.AssertionError: null at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertFalse(Assert.java:64) at org.junit.Assert.assertFalse(Assert.java:74) at org.apache.hadoop.hbase.master.TestMasterFailoverBalancerPersistence.killActiveAndWaitForNewActive(TestMasterFailoverBalancerPersistence.java:99) at org.apache.hadoop.hbase.master.TestMasterFailoverBalancerPersistence.testMasterFailoverBalancerPersistence(TestMasterFailoverBalancerPersistence.java:72) Standard Output Formatting using clusterid: testClusterID Standard Error 2017-02-23 21:54:37,888 INFO [main] hbase.ResourceChecker(147): before: master.TestMasterFailoverBalancerPersistence#testMasterFailoverBalancerPersistence Thread=4, OpenFileDescriptor=213, MaxFileDescriptor=1024000, SystemLoadAverage=795, ProcessCount=11, AvailableMemoryMB=17617, ConnectionCount=0 2017-02-23 21:54:38,197 INFO [Time-limited test] hbase.HBaseTestingUtility(1003): Starting up minicluster with 3 master(s) and 1 regionserver(s) and 1 datanode(s) 2017-02-23 21:54:38,220 INFO [Time-limited test] hbase.HBaseTestingUtility(471): Created new mini-cluster data directory: /grid/0/nobody/workspace/build-support/SOURCES/hbase/hbase-server/target/test-data/daf339ed-dbf7-4104-9a70-1cba0ef96dcb/dfscluster_ecb78160-5968-4c10-a41e-a4687da03fe5, deleteOnExit=true 2017-02-23 21:54:38,221 INFO [Time-limited test] hbase.HBaseTestingUtility(718): Setting test.cache.data to /grid/0/nobody/workspace/build-support/SOURCES/hbase/hbase-server/target/test-data/daf339ed-dbf7-4104-9a70-1cba0ef96dcb/cache_data in system properties and HBase conf 2017-02-23 21:54:38,221 INFO [Time-limited test] hbase.HBaseTestingUtility(718): Setting hadoop.tmp.dir to /grid/0/nobody/workspace/build-support/SOURCES/hbase/hbase-server/target/test-data/daf339ed-dbf7-4104-9a70-1cba0ef96dcb/hadoop_tmp in system properties and HBase conf 2017-02-23 21:54:38,222 INFO [Time-limited test] hbase.HBaseTestingUtility(718): Setting hadoop.log.dir to /grid/0/nobody/workspace/build-support/SOURCES/hbase/hbase-server/target/test-data/daf339ed-dbf7-4104-9a70-1cba0ef96dcb/hadoop_logs in system properties and HBase conf 2017-02-23 21:54:38,222 INFO [Time-limited test] hbase.HBaseTestingUtility(718): Setting mapreduce.cluster.local.dir to /grid/0/nobody/workspace/build-support/SOURCES/hbase/hbase-server/target/test-data/daf339ed-dbf7-4104-9a70-1cba0ef96dcb/mapred_local in system properties and HBase conf 2017-02-23 21:54:38,223 INFO [Time-limited test] hbase.HBaseTestingUtility(718): Setting mapreduce.cluster.temp.dir to /grid/0/nobody/workspace/build-support/SOURCES/hbase/hbase-server/target/test-data/daf339ed-dbf7-4104-9a70-1cba0ef96dcb/mapred_temp in system properties and HBase conf 2017-02-23 21:54:38,223 INFO [Time-limited test] hbase.HBaseTestingUtility(709): read short circuit is OFF 2017-02-23 21:54:38,352 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2017-02-23 21:54:38,750 DEBUG [Time-limited test] fs.HFileSystem(225): The file system is not a DistributedFileSystem. Skipping on block location reordering 2017-02-23 21:54:39,537 WARN [Time-limited test] impl.MetricsConfig(125): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2017-02-23 21:54:39,717 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2017-02-23 21:54:39,789 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26.hwx 2017-02-23 21:54:39,824 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/grid/0/nobody/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.7.3.2.6.0.0-SNAPSHOT/hadoop-hdfs-2.7.3.2.6.0.0-SNAPSHOT-tests.jar!/webapps/hdfs to /tmp/Jetty_localhost_52864_hdfs____mb8c0z/webapp 2017-02-23 21:54:40,004 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:52864 2017-02-23 21:54:40,451 INFO [org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@4658a66c] blockmanagement.BlockManager(1594): BLOCK* neededReplications = 0, pendingReplications = 0. 2017-02-23 21:54:40,742 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26.hwx 2017-02-23 21:54:40,746 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/grid/0/nobody/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.7.3.2.6.0.0-SNAPSHOT/hadoop-hdfs-2.7.3.2.6.0.0-SNAPSHOT-tests.jar!/webapps/datanode to /tmp/Jetty_localhost_43576_datanode____otuqfw/webapp 2017-02-23 21:54:40,868 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43576 2017-02-23 21:54:41,662 INFO [Block report processor] blockmanagement.BlockManager(1978): BLOCK* processReport: from storage DS-f2877bbf-9a69-49bf-b5b8-d9fc7b715f48 node DatanodeRegistration(127.0.0.1:51976, datanodeUuid=fe84228d-7aa1-410a-b8ab-65c5af7303f5, infoPort=44340, infoSecurePort=0, ipcPort=36258, storageInfo=lv=-56;cid=testClusterID;nsid=679203390;c=0), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2017-02-23 21:54:41,663 INFO [Block report processor] blockmanagement.BlockManager(1978): BLOCK* processReport: from storage DS-a459f8e7-1aa9-4e44-95c6-1c270f8276f4 node DatanodeRegistration(127.0.0.1:51976, datanodeUuid=fe84228d-7aa1-410a-b8ab-65c5af7303f5, infoPort=44340, infoSecurePort=0, ipcPort=36258, storageInfo=lv=-56;cid=testClusterID;nsid=679203390;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2017-02-23 21:54:41,733 INFO [Time-limited test] zookeeper.MiniZooKeeperCluster(276): Started MiniZooKeeperCluster and ran successful 'stat' on client port=49322 2017-02-23 21:54:41,753 INFO [Time-limited test] fs.HFileSystem(256): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2017-02-23 21:54:41,759 INFO [Time-limited test] fs.HFileSystem(256): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2017-02-23 21:54:42,502 INFO [Time-limited test] util.FSUtils(763): Created version file at hdfs://localhost:59749/user/nobody/test-data/3c09ad53-b369-4742-aa28-6ea8cf270e8b with version=8 2017-02-23 21:54:42,571 DEBUG [Time-limited test] impl.BackupManager(160): Added region procedure manager: org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager 2017-02-23 21:54:42,748 INFO [Time-limited test] client.ConnectionUtils(104): master/xxx/xxx:0 server-side HConnection retries=350 2017-02-23 21:54:42,759 INFO [Time-limited test] ipc.SimpleRpcScheduler(196): Using fifo as user call queue, count=1 2017-02-23 21:54:42,766 INFO [Time-limited test] ipc.RpcServer$Listener(608): master/xxx/xxx:0: started 10 reader(s). 2017-02-23 21:54:42,802 INFO [Time-limited test] hfile.CacheConfig(530): Allocating LruBlockCache size=995.58 MB, blockSize=64 KB 2017-02-23 21:54:42,808 DEBUG [Time-limited test] hfile.CacheConfig(544): Trying to use Internal l2 cache 2017-02-23 21:54:42,809 INFO [Time-limited test] hfile.CacheConfig(278): blockCache=LruBlockCache{blockCount=0, currentSize=1071760, freeSize=1042864304, maxSize=1043936064, heapSize=1071760, minSize=991739264, minFactor=0.95, multiSize=495869632, multiFactor=0.5, singleSize=247934816, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 2017-02-23 21:54:42,810 INFO [Time-limited test] hfile.CacheConfig(278): blockCache=LruBlockCache{blockCount=0, currentSize=1071760, freeSize=1042864304, maxSize=1043936064, heapSize=1071760, minSize=991739264, minFactor=0.95, multiSize=495869632, multiFactor=0.5, singleSize=247934816, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 2017-02-23 21:54:42,815 INFO [Time-limited test] mob.MobFileCache(121): MobFileCache is initialized, and the cache size is 1000 2017-02-23 21:54:42,819 INFO [Time-limited test] fs.HFileSystem(256): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2017-02-23 21:54:42,828 INFO [Time-limited test] fs.HFileSystem(256): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2017-02-23 21:54:42,887 INFO [Time-limited test] zookeeper.RecoverableZooKeeper(120): Process identifier=master:53271 connecting to ZooKeeper ensemble=localhost:49322 2017-02-23 21:54:42,931 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:532710x0, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2017-02-23 21:54:42,932 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(578): master:53271-0x15a6cf813ab0000 connected 2017-02-23 21:54:42,979 DEBUG [Time-limited test] zookeeper.ZKUtil(369): master:53271-0x15a6cf813ab0000, quorum=localhost:49322, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2017-02-23 21:54:42,980 DEBUG [Time-limited test] zookeeper.ZKUtil(369): master:53271-0x15a6cf813ab0000, quorum=localhost:49322, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2017-02-23 21:54:42,982 INFO [RpcServer.responder] ipc.RpcServer$Responder(927): RpcServer.responder: starting 2017-02-23 21:54:42,983 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=0,queue=0,port=53271 2017-02-23 21:54:42,983 INFO [RpcServer.listener,port=53271] ipc.RpcServer$Listener(739): RpcServer.listener,port=53271: starting 2017-02-23 21:54:42,983 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=1,queue=0,port=53271 2017-02-23 21:54:42,984 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=2,queue=0,port=53271 2017-02-23 21:54:42,985 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=3,queue=0,port=53271 2017-02-23 21:54:42,985 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=4,queue=0,port=53271 2017-02-23 21:54:42,986 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=5,queue=0,port=53271 2017-02-23 21:54:42,986 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=0,queue=0,port=53271 2017-02-23 21:54:42,987 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=1,queue=1,port=53271 2017-02-23 21:54:42,987 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=2,queue=0,port=53271 2017-02-23 21:54:42,987 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=3,queue=1,port=53271 2017-02-23 21:54:42,988 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=4,queue=0,port=53271 2017-02-23 21:54:42,988 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=53271 2017-02-23 21:54:42,989 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=6,queue=0,port=53271 2017-02-23 21:54:42,990 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=7,queue=1,port=53271 2017-02-23 21:54:42,990 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=8,queue=0,port=53271 2017-02-23 21:54:42,991 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=9,queue=1,port=53271 2017-02-23 21:54:42,991 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=10,queue=0,port=53271 2017-02-23 21:54:42,992 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=11,queue=1,port=53271 2017-02-23 21:54:42,992 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=12,queue=0,port=53271 2017-02-23 21:54:42,993 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=13,queue=1,port=53271 2017-02-23 21:54:42,994 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=14,queue=0,port=53271 2017-02-23 21:54:42,994 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=15,queue=1,port=53271 2017-02-23 21:54:42,994 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=16,queue=0,port=53271 2017-02-23 21:54:42,995 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=17,queue=1,port=53271 2017-02-23 21:54:42,995 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=18,queue=0,port=53271 2017-02-23 21:54:42,995 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=19,queue=1,port=53271 2017-02-23 21:54:42,996 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.replication.handler=0,queue=0,port=53271 2017-02-23 21:54:42,996 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.replication.handler=1,queue=0,port=53271 2017-02-23 21:54:42,996 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.replication.handler=2,queue=0,port=53271 2017-02-23 21:54:43,001 INFO [Time-limited test] master.HMaster(429): hbase.rootdir=hdfs://localhost:59749/user/nobody/test-data/3c09ad53-b369-4742-aa28-6ea8cf270e8b, hbase.cluster.distributed=false 2017-02-23 21:54:43,011 DEBUG [Time-limited test] impl.BackupManager(136): Added log cleaner: org.apache.hadoop.hbase.backup.master.BackupLogCleaner 2017-02-23 21:54:43,011 DEBUG [Time-limited test] impl.BackupManager(137): Added master procedure manager: org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager 2017-02-23 21:54:43,011 DEBUG [Time-limited test] impl.BackupManager(138): Added master observer: org.apache.hadoop.hbase.backup.master.BackupController 2017-02-23 21:54:43,026 INFO [Time-limited test] master.HMaster(1897): Adding backup master ZNode /hbase/backup-masters/xxx,53271,1487886882791 2017-02-23 21:54:43,037 DEBUG [Time-limited test] zookeeper.ZKUtil(367): master:53271-0x15a6cf813ab0000, quorum=localhost:49322, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/xxx,53271,1487886882791 2017-02-23 21:54:43,041 DEBUG [Time-limited test] impl.BackupManager(160): Added region procedure manager: org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager 2017-02-23 21:54:43,044 INFO [Time-limited test] client.ConnectionUtils(104): master/xxx/xxx:0 server-side HConnection retries=350 2017-02-23 21:54:43,045 INFO [Time-limited test] ipc.SimpleRpcScheduler(196): Using fifo as user call queue, count=1 2017-02-23 21:54:43,049 INFO [Time-limited test] ipc.RpcServer$Listener(608): master/xxx/xxx:0: started 10 reader(s). 2017-02-23 21:54:43,051 INFO [Time-limited test] hfile.CacheConfig(278): blockCache=LruBlockCache{blockCount=0, currentSize=1071760, freeSize=1042864304, maxSize=1043936064, heapSize=1071760, minSize=991739264, minFactor=0.95, multiSize=495869632, multiFactor=0.5, singleSize=247934816, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 2017-02-23 21:54:43,051 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:53271-0x15a6cf813ab0000, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2017-02-23 21:54:43,051 INFO [Time-limited test] hfile.CacheConfig(278): blockCache=LruBlockCache{blockCount=0, currentSize=1071760, freeSize=1042864304, maxSize=1043936064, heapSize=1071760, minSize=991739264, minFactor=0.95, multiSize=495869632, multiFactor=0.5, singleSize=247934816, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 2017-02-23 21:54:43,052 DEBUG [xxx:53271.activeMasterManager] zookeeper.ZKUtil(367): master:53271-0x15a6cf813ab0000, quorum=localhost:49322, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2017-02-23 21:54:43,053 INFO [xxx:53271.activeMasterManager] master.ActiveMasterManager(170): Deleting ZNode for /hbase/backup-masters/xxx,53271,1487886882791 from backup master directory 2017-02-23 21:54:43,055 INFO [Time-limited test] fs.HFileSystem(256): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2017-02-23 21:54:43,056 DEBUG [Time-limited test-EventThread] zookeeper.ZKUtil(367): master:53271-0x15a6cf813ab0000, quorum=localhost:49322, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2017-02-23 21:54:43,056 DEBUG [Time-limited test-EventThread] master.ActiveMasterManager(126): A master is now available 2017-02-23 21:54:43,057 WARN [xxx:53271.activeMasterManager] hbase.ZNodeClearer(58): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2017-02-23 21:54:43,057 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:53271-0x15a6cf813ab0000, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/xxx,53271,1487886882791 2017-02-23 21:54:43,057 INFO [xxx:53271.activeMasterManager] master.ActiveMasterManager(179): Registered Active Master=xxx,53271,1487886882791 2017-02-23 21:54:43,059 INFO [Time-limited test] fs.HFileSystem(256): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2017-02-23 21:54:43,062 INFO [Time-limited test] zookeeper.RecoverableZooKeeper(120): Process identifier=master:56195 connecting to ZooKeeper ensemble=localhost:49322 2017-02-23 21:54:43,065 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:561950x0, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2017-02-23 21:54:43,066 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(578): master:56195-0x15a6cf813ab0001 connected 2017-02-23 21:54:43,068 DEBUG [Time-limited test] zookeeper.RecoverableZooKeeper(584): Node /hbase already exists 2017-02-23 21:54:43,076 DEBUG [Time-limited test] zookeeper.ZKUtil(367): master:56195-0x15a6cf813ab0001, quorum=localhost:49322, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2017-02-23 21:54:43,077 DEBUG [Time-limited test] zookeeper.ZKUtil(369): master:56195-0x15a6cf813ab0001, quorum=localhost:49322, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2017-02-23 21:54:43,078 INFO [RpcServer.responder] ipc.RpcServer$Responder(927): RpcServer.responder: starting 2017-02-23 21:54:43,078 INFO [RpcServer.listener,port=56195] ipc.RpcServer$Listener(739): RpcServer.listener,port=56195: starting 2017-02-23 21:54:43,078 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=0,queue=0,port=56195 2017-02-23 21:54:43,079 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=1,queue=0,port=56195 2017-02-23 21:54:43,080 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=2,queue=0,port=56195 2017-02-23 21:54:43,080 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=3,queue=0,port=56195 2017-02-23 21:54:43,080 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=4,queue=0,port=56195 2017-02-23 21:54:43,081 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=5,queue=0,port=56195 2017-02-23 21:54:43,081 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=0,queue=0,port=56195 2017-02-23 21:54:43,081 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=1,queue=1,port=56195 2017-02-23 21:54:43,082 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=2,queue=0,port=56195 2017-02-23 21:54:43,082 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=3,queue=1,port=56195 2017-02-23 21:54:43,082 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=4,queue=0,port=56195 2017-02-23 21:54:43,083 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=56195 2017-02-23 21:54:43,083 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=6,queue=0,port=56195 2017-02-23 21:54:43,083 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=7,queue=1,port=56195 2017-02-23 21:54:43,084 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=8,queue=0,port=56195 2017-02-23 21:54:43,084 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=9,queue=1,port=56195 2017-02-23 21:54:43,084 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=10,queue=0,port=56195 2017-02-23 21:54:43,085 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=11,queue=1,port=56195 2017-02-23 21:54:43,085 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=12,queue=0,port=56195 2017-02-23 21:54:43,085 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=13,queue=1,port=56195 2017-02-23 21:54:43,086 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=14,queue=0,port=56195 2017-02-23 21:54:43,086 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=15,queue=1,port=56195 2017-02-23 21:54:43,086 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=16,queue=0,port=56195 2017-02-23 21:54:43,087 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=17,queue=1,port=56195 2017-02-23 21:54:43,087 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=18,queue=0,port=56195 2017-02-23 21:54:43,087 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=19,queue=1,port=56195 2017-02-23 21:54:43,088 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.replication.handler=0,queue=0,port=56195 2017-02-23 21:54:43,088 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.replication.handler=1,queue=0,port=56195 2017-02-23 21:54:43,088 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.replication.handler=2,queue=0,port=56195 2017-02-23 21:54:43,089 INFO [Time-limited test] master.HMaster(429): hbase.rootdir=hdfs://localhost:59749/user/nobody/test-data/3c09ad53-b369-4742-aa28-6ea8cf270e8b, hbase.cluster.distributed=false 2017-02-23 21:54:43,089 DEBUG [Time-limited test] impl.BackupManager(136): Added log cleaner: org.apache.hadoop.hbase.backup.master.BackupLogCleaner 2017-02-23 21:54:43,089 DEBUG [Time-limited test] impl.BackupManager(137): Added master procedure manager: org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager 2017-02-23 21:54:43,089 DEBUG [Time-limited test] impl.BackupManager(138): Added master observer: org.apache.hadoop.hbase.backup.master.BackupController 2017-02-23 21:54:43,090 INFO [Time-limited test] master.HMaster(1897): Adding backup master ZNode /hbase/backup-masters/xxx,56195,1487886883050 2017-02-23 21:54:43,092 DEBUG [Time-limited test] zookeeper.ZKUtil(367): master:56195-0x15a6cf813ab0001, quorum=localhost:49322, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/xxx,56195,1487886883050 2017-02-23 21:54:43,094 DEBUG [Time-limited test] impl.BackupManager(160): Added region procedure manager: org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager 2017-02-23 21:54:43,095 DEBUG [xxx:56195.activeMasterManager] zookeeper.RecoverableZooKeeper(584): Node /hbase/master already exists 2017-02-23 21:54:43,096 DEBUG [xxx:56195.activeMasterManager] zookeeper.ZKUtil(367): master:56195-0x15a6cf813ab0001, quorum=localhost:49322, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2017-02-23 21:54:43,096 INFO [xxx:56195.activeMasterManager] master.ActiveMasterManager(216): Another master is the active master, xxx,53271,1487886882791; waiting to become the next active master 2017-02-23 21:54:43,097 INFO [Time-limited test] client.ConnectionUtils(104): master/xxx/xxx:0 server-side HConnection retries=350 2017-02-23 21:54:43,097 INFO [Time-limited test] ipc.SimpleRpcScheduler(196): Using fifo as user call queue, count=1 2017-02-23 21:54:43,100 INFO [Time-limited test] ipc.RpcServer$Listener(608): master/xxx/xxx:0: started 10 reader(s). 2017-02-23 21:54:43,101 INFO [Time-limited test] hfile.CacheConfig(278): blockCache=LruBlockCache{blockCount=0, currentSize=1071760, freeSize=1042864304, maxSize=1043936064, heapSize=1071760, minSize=991739264, minFactor=0.95, multiSize=495869632, multiFactor=0.5, singleSize=247934816, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 2017-02-23 21:54:43,101 INFO [Time-limited test] hfile.CacheConfig(278): blockCache=LruBlockCache{blockCount=0, currentSize=1071760, freeSize=1042864304, maxSize=1043936064, heapSize=1071760, minSize=991739264, minFactor=0.95, multiSize=495869632, multiFactor=0.5, singleSize=247934816, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 2017-02-23 21:54:43,104 INFO [Time-limited test] fs.HFileSystem(256): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2017-02-23 21:54:43,108 INFO [Time-limited test] fs.HFileSystem(256): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2017-02-23 21:54:43,110 INFO [Time-limited test] zookeeper.RecoverableZooKeeper(120): Process identifier=master:55048 connecting to ZooKeeper ensemble=localhost:49322 2017-02-23 21:54:43,257 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:550480x0, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2017-02-23 21:54:43,257 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(578): master:55048-0x15a6cf813ab0002 connected 2017-02-23 21:54:43,278 DEBUG [Time-limited test] zookeeper.RecoverableZooKeeper(584): Node /hbase already exists 2017-02-23 21:54:43,284 DEBUG [Time-limited test] zookeeper.ZKUtil(367): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2017-02-23 21:54:43,285 DEBUG [Time-limited test] zookeeper.ZKUtil(369): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2017-02-23 21:54:43,286 INFO [RpcServer.responder] ipc.RpcServer$Responder(927): RpcServer.responder: starting 2017-02-23 21:54:43,286 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=0,queue=0,port=55048 2017-02-23 21:54:43,286 INFO [RpcServer.listener,port=55048] ipc.RpcServer$Listener(739): RpcServer.listener,port=55048: starting 2017-02-23 21:54:43,286 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=1,queue=0,port=55048 2017-02-23 21:54:43,287 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=2,queue=0,port=55048 2017-02-23 21:54:43,287 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=3,queue=0,port=55048 2017-02-23 21:54:43,287 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=4,queue=0,port=55048 2017-02-23 21:54:43,288 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=5,queue=0,port=55048 2017-02-23 21:54:43,288 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=0,queue=0,port=55048 2017-02-23 21:54:43,288 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=1,queue=1,port=55048 2017-02-23 21:54:43,289 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=2,queue=0,port=55048 2017-02-23 21:54:43,289 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=3,queue=1,port=55048 2017-02-23 21:54:43,289 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=4,queue=0,port=55048 2017-02-23 21:54:43,289 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=55048 2017-02-23 21:54:43,290 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=6,queue=0,port=55048 2017-02-23 21:54:43,290 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=7,queue=1,port=55048 2017-02-23 21:54:43,290 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=8,queue=0,port=55048 2017-02-23 21:54:43,291 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=9,queue=1,port=55048 2017-02-23 21:54:43,291 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=10,queue=0,port=55048 2017-02-23 21:54:43,291 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=11,queue=1,port=55048 2017-02-23 21:54:43,291 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=12,queue=0,port=55048 2017-02-23 21:54:43,292 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=13,queue=1,port=55048 2017-02-23 21:54:43,292 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=14,queue=0,port=55048 2017-02-23 21:54:43,292 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=15,queue=1,port=55048 2017-02-23 21:54:43,293 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=16,queue=0,port=55048 2017-02-23 21:54:43,293 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=17,queue=1,port=55048 2017-02-23 21:54:43,293 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=18,queue=0,port=55048 2017-02-23 21:54:43,294 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=19,queue=1,port=55048 2017-02-23 21:54:43,294 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.replication.handler=0,queue=0,port=55048 2017-02-23 21:54:43,294 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.replication.handler=1,queue=0,port=55048 2017-02-23 21:54:43,294 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.replication.handler=2,queue=0,port=55048 2017-02-23 21:54:43,295 INFO [Time-limited test] master.HMaster(429): hbase.rootdir=hdfs://localhost:59749/user/nobody/test-data/3c09ad53-b369-4742-aa28-6ea8cf270e8b, hbase.cluster.distributed=false 2017-02-23 21:54:43,295 DEBUG [Time-limited test] impl.BackupManager(136): Added log cleaner: org.apache.hadoop.hbase.backup.master.BackupLogCleaner 2017-02-23 21:54:43,295 DEBUG [Time-limited test] impl.BackupManager(137): Added master procedure manager: org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager 2017-02-23 21:54:43,295 DEBUG [Time-limited test] impl.BackupManager(138): Added master observer: org.apache.hadoop.hbase.backup.master.BackupController 2017-02-23 21:54:43,296 INFO [Time-limited test] master.HMaster(1897): Adding backup master ZNode /hbase/backup-masters/xxx,55048,1487886883101 2017-02-23 21:54:43,306 DEBUG [Time-limited test] zookeeper.ZKUtil(367): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/xxx,55048,1487886883101 2017-02-23 21:54:43,311 DEBUG [xxx:55048.activeMasterManager] zookeeper.RecoverableZooKeeper(584): Node /hbase/master already exists 2017-02-23 21:54:43,312 DEBUG [xxx:55048.activeMasterManager] zookeeper.ZKUtil(367): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2017-02-23 21:54:43,313 INFO [xxx:55048.activeMasterManager] master.ActiveMasterManager(216): Another master is the active master, xxx,53271,1487886882791; waiting to become the next active master 2017-02-23 21:54:43,363 DEBUG [Time-limited test] impl.BackupManager(160): Added region procedure manager: org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager 2017-02-23 21:54:43,365 INFO [Time-limited test] client.ConnectionUtils(104): regionserver/xxx/xxx:0 server-side HConnection retries=350 2017-02-23 21:54:43,365 INFO [Time-limited test] ipc.SimpleRpcScheduler(196): Using fifo as user call queue, count=1 2017-02-23 21:54:43,370 INFO [Time-limited test] ipc.RpcServer$Listener(608): regionserver/xxx/xxx:0: started 10 reader(s). 2017-02-23 21:54:43,378 INFO [Time-limited test] hfile.CacheConfig(278): blockCache=LruBlockCache{blockCount=0, currentSize=1071760, freeSize=1042864304, maxSize=1043936064, heapSize=1071760, minSize=991739264, minFactor=0.95, multiSize=495869632, multiFactor=0.5, singleSize=247934816, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 2017-02-23 21:54:43,379 INFO [Time-limited test] hfile.CacheConfig(278): blockCache=LruBlockCache{blockCount=0, currentSize=1071760, freeSize=1042864304, maxSize=1043936064, heapSize=1071760, minSize=991739264, minFactor=0.95, multiSize=495869632, multiFactor=0.5, singleSize=247934816, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 2017-02-23 21:54:43,382 INFO [Time-limited test] fs.HFileSystem(256): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2017-02-23 21:54:43,386 INFO [Time-limited test] fs.HFileSystem(256): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2017-02-23 21:54:43,388 INFO [Time-limited test] zookeeper.RecoverableZooKeeper(120): Process identifier=regionserver:54602 connecting to ZooKeeper ensemble=localhost:49322 2017-02-23 21:54:43,392 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): regionserver:546020x0, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2017-02-23 21:54:43,393 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(578): regionserver:54602-0x15a6cf813ab0003 connected 2017-02-23 21:54:43,393 DEBUG [Time-limited test] zookeeper.ZKUtil(367): regionserver:54602-0x15a6cf813ab0003, quorum=localhost:49322, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2017-02-23 21:54:43,394 DEBUG [Time-limited test] zookeeper.ZKUtil(369): regionserver:54602-0x15a6cf813ab0003, quorum=localhost:49322, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2017-02-23 21:54:43,395 INFO [RpcServer.responder] ipc.RpcServer$Responder(927): RpcServer.responder: starting 2017-02-23 21:54:43,395 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=0,queue=0,port=54602 2017-02-23 21:54:43,395 INFO [RpcServer.listener,port=54602] ipc.RpcServer$Listener(739): RpcServer.listener,port=54602: starting 2017-02-23 21:54:43,395 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=1,queue=0,port=54602 2017-02-23 21:54:43,396 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=2,queue=0,port=54602 2017-02-23 21:54:43,396 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=3,queue=0,port=54602 2017-02-23 21:54:43,397 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=4,queue=0,port=54602 2017-02-23 21:54:43,397 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.default.handler=5,queue=0,port=54602 2017-02-23 21:54:43,397 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=0,queue=0,port=54602 2017-02-23 21:54:43,398 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=1,queue=1,port=54602 2017-02-23 21:54:43,398 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=2,queue=0,port=54602 2017-02-23 21:54:43,398 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=3,queue=1,port=54602 2017-02-23 21:54:43,398 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=4,queue=0,port=54602 2017-02-23 21:54:43,399 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=54602 2017-02-23 21:54:43,399 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=6,queue=0,port=54602 2017-02-23 21:54:43,399 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=7,queue=1,port=54602 2017-02-23 21:54:43,400 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=8,queue=0,port=54602 2017-02-23 21:54:43,400 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=9,queue=1,port=54602 2017-02-23 21:54:43,400 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=10,queue=0,port=54602 2017-02-23 21:54:43,400 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=11,queue=1,port=54602 2017-02-23 21:54:43,401 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=12,queue=0,port=54602 2017-02-23 21:54:43,401 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=13,queue=1,port=54602 2017-02-23 21:54:43,401 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=14,queue=0,port=54602 2017-02-23 21:54:43,402 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=15,queue=1,port=54602 2017-02-23 21:54:43,402 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=16,queue=0,port=54602 2017-02-23 21:54:43,402 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=17,queue=1,port=54602 2017-02-23 21:54:43,403 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=18,queue=0,port=54602 2017-02-23 21:54:43,403 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.priority.handler=19,queue=1,port=54602 2017-02-23 21:54:43,403 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.replication.handler=0,queue=0,port=54602 2017-02-23 21:54:43,404 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.replication.handler=1,queue=0,port=54602 2017-02-23 21:54:43,404 DEBUG [Time-limited test] ipc.RpcExecutor(129): Started RpcServer.FifoWFPBQ.replication.handler=2,queue=0,port=54602 2017-02-23 21:54:43,412 INFO [Time-limited test] http.HttpRequestLog(69): Http request log for http.requests.regionserver is not defined 2017-02-23 21:54:43,413 INFO [Time-limited test] http.HttpServer(852): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2017-02-23 21:54:43,415 INFO [Time-limited test] http.HttpServer(830): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2017-02-23 21:54:43,415 INFO [Time-limited test] http.HttpServer(837): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2017-02-23 21:54:43,416 INFO [Time-limited test] http.HttpServer(837): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2017-02-23 21:54:43,420 INFO [Time-limited test] http.HttpServer(1083): Jetty bound to port 55849 2017-02-23 21:54:43,420 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26.hwx 2017-02-23 21:54:43,452 INFO [org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@4658a66c] blockmanagement.BlockManager(1594): BLOCK* neededReplications = 0, pendingReplications = 0. 2017-02-23 21:54:43,480 INFO [Time-limited test] log.Slf4jLog(67): Started SelectChannelConnector@0.0.0.0:55849 2017-02-23 21:54:43,506 INFO [M:1;xxx:56195] zookeeper.RecoverableZooKeeper(120): Process identifier=hconnection-0x7b139ebd connecting to ZooKeeper ensemble=localhost:49322 2017-02-23 21:54:43,506 INFO [M:0;xxx:53271] zookeeper.RecoverableZooKeeper(120): Process identifier=hconnection-0x451ac96b connecting to ZooKeeper ensemble=localhost:49322 2017-02-23 21:54:43,506 INFO [RS:0;xxx:54602] zookeeper.RecoverableZooKeeper(120): Process identifier=hconnection-0x619db33e connecting to ZooKeeper ensemble=localhost:49322 2017-02-23 21:54:43,506 INFO [M:2;xxx:55048] zookeeper.RecoverableZooKeeper(120): Process identifier=hconnection-0x2e6d1362 connecting to ZooKeeper ensemble=localhost:49322 2017-02-23 21:54:43,512 DEBUG [M:0;xxx:53271-EventThread] zookeeper.ZooKeeperWatcher(515): hconnection-0x451ac96b0x0, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2017-02-23 21:54:43,512 DEBUG [M:0;xxx:53271-EventThread] zookeeper.ZooKeeperWatcher(578): hconnection-0x451ac96b-0x15a6cf813ab0004 connected 2017-02-23 21:54:43,513 DEBUG [M:1;xxx:56195-EventThread] zookeeper.ZooKeeperWatcher(515): hconnection-0x7b139ebd0x0, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2017-02-23 21:54:43,513 DEBUG [M:1;xxx:56195-EventThread] zookeeper.ZooKeeperWatcher(578): hconnection-0x7b139ebd-0x15a6cf813ab0005 connected 2017-02-23 21:54:43,513 DEBUG [M:2;xxx:55048-EventThread] zookeeper.ZooKeeperWatcher(515): hconnection-0x2e6d13620x0, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2017-02-23 21:54:43,514 DEBUG [M:2;xxx:55048-EventThread] zookeeper.ZooKeeperWatcher(578): hconnection-0x2e6d1362-0x15a6cf813ab0006 connected 2017-02-23 21:54:43,514 DEBUG [RS:0;xxx:54602-EventThread] zookeeper.ZooKeeperWatcher(515): hconnection-0x619db33e0x0, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2017-02-23 21:54:43,514 DEBUG [RS:0;xxx:54602-EventThread] zookeeper.ZooKeeperWatcher(578): hconnection-0x619db33e-0x15a6cf813ab0007 connected 2017-02-23 21:54:43,514 INFO [M:0;xxx:53271] client.ZooKeeperRegistry(107): ClusterId read in ZooKeeper is null 2017-02-23 21:54:43,514 INFO [M:1;xxx:56195] client.ZooKeeperRegistry(107): ClusterId read in ZooKeeper is null 2017-02-23 21:54:43,515 INFO [RS:0;xxx:54602] client.ZooKeeperRegistry(107): ClusterId read in ZooKeeper is null 2017-02-23 21:54:43,515 DEBUG [M:0;xxx:53271] client.ConnectionManager$HConnectionImplementation(892): clusterid came back null, using default default-cluster 2017-02-23 21:54:43,515 INFO [M:2;xxx:55048] client.ZooKeeperRegistry(107): ClusterId read in ZooKeeper is null 2017-02-23 21:54:43,515 DEBUG [M:2;xxx:55048] client.ConnectionManager$HConnectionImplementation(892): clusterid came back null, using default default-cluster 2017-02-23 21:54:43,515 DEBUG [RS:0;xxx:54602] client.ConnectionManager$HConnectionImplementation(892): clusterid came back null, using default default-cluster 2017-02-23 21:54:43,515 DEBUG [M:1;xxx:56195] client.ConnectionManager$HConnectionImplementation(892): clusterid came back null, using default default-cluster 2017-02-23 21:54:43,522 DEBUG [M:2;xxx:55048] ipc.AbstractRpcClient(105): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6e7d7591, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2017-02-23 21:54:43,522 DEBUG [M:1;xxx:56195] ipc.AbstractRpcClient(105): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6eb1bb42, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2017-02-23 21:54:43,522 DEBUG [RS:0;xxx:54602] ipc.AbstractRpcClient(105): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@75c8d05d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2017-02-23 21:54:43,522 DEBUG [M:0;xxx:53271] ipc.AbstractRpcClient(105): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@52e7c614, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2017-02-23 21:54:43,571 DEBUG [xxx:53271.activeMasterManager] util.FSUtils(915): Created cluster ID file at hdfs://localhost:59749/user/nobody/test-data/3c09ad53-b369-4742-aa28-6ea8cf270e8b/hbase.id with ID: 33c12452-369c-4de8-8dd0-16f76986b10f 2017-02-23 21:54:43,612 INFO [xxx:53271.activeMasterManager] master.MasterFileSystem(560): BOOTSTRAP: creating hbase:meta region 2017-02-23 21:54:43,614 INFO [xxx:53271.activeMasterManager] regionserver.HRegion(6360): creating HRegion hbase:meta HTD == 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}, {NAME => 'info', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '10', TTL => 'FOREVER', MIN_VERSIONS => '0', CACHE_DATA_IN_L1 => 'true', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '8192', I ...[truncated 151761 chars]... erver=xxx,54602,1487886883378} 2017-02-23 21:54:48,486 DEBUG [xxx:56195.activeMasterManager] master.RegionStates(505): Onlined 1588230740 on xxx,54602,1487886883378 {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2017-02-23 21:54:48,487 INFO [xxx:56195.activeMasterManager] master.ServerManager(684): AssignmentManager hasn't finished failover cleanup; waiting 2017-02-23 21:54:48,488 INFO [xxx:56195.activeMasterManager] master.HMaster(1054): hbase:meta with replicaId 0 assigned=0, rit=false, location=xxx,54602,1487886883378 2017-02-23 21:54:48,492 DEBUG [RS_OPEN_PRIORITY_REGION-xxx:54602-1] wal.WALSplitter(720): Wrote region seqId=hdfs://localhost:59749/user/nobody/test-data/3c09ad53-b369-4742-aa28-6ea8cf270e8b/data/hbase/backup/3c1c23619d058b12671d9681e36bf99c/recovered.edits/2.seqid to file, newSeqId=2, maxSeqId=0 2017-02-23 21:54:48,492 INFO [RS_OPEN_PRIORITY_REGION-xxx:54602-1] regionserver.HRegion(924): Onlined 3c1c23619d058b12671d9681e36bf99c; next sequenceid=2 2017-02-23 21:54:48,494 INFO [xxx:56195.activeMasterManager] hbase.MetaMigrationConvertingToPB(131): META already up-to date with PB serialization 2017-02-23 21:54:48,496 DEBUG [RS_OPEN_PRIORITY_REGION-xxx:54602-1] zookeeper.ZKAssign(643): regionserver:54602-0x15a6cf813ab0003, quorum=localhost:49322, baseZNode=/hbase Attempting to retransition opening state of node 3c1c23619d058b12671d9681e36bf99c 2017-02-23 21:54:48,498 INFO [PostOpenDeployTasks:3c1c23619d058b12671d9681e36bf99c] regionserver.HRegionServer(2025): Post open deploy tasks for hbase:backup,,1487886887259.3c1c23619d058b12671d9681e36bf99c. 2017-02-23 21:54:48,515 INFO [PostOpenDeployTasks:3c1c23619d058b12671d9681e36bf99c] hbase.MetaTableAccessor(1420): Updated row hbase:backup,,1487886887259.3c1c23619d058b12671d9681e36bf99c. with server=xxx,54602,1487886883378 2017-02-23 21:54:48,515 DEBUG [PostOpenDeployTasks:3c1c23619d058b12671d9681e36bf99c] regionserver.HRegionServer(2059): Finished post open deploy task for hbase:backup,,1487886887259.3c1c23619d058b12671d9681e36bf99c. 2017-02-23 21:54:48,516 DEBUG [RS_OPEN_PRIORITY_REGION-xxx:54602-1] zookeeper.ZKAssign(805): regionserver:54602-0x15a6cf813ab0003, quorum=localhost:49322, baseZNode=/hbase Transitioning 3c1c23619d058b12671d9681e36bf99c from RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENED 2017-02-23 21:54:48,518 DEBUG [RS_OPEN_PRIORITY_REGION-xxx:54602-1] zookeeper.ZKAssign(880): regionserver:54602-0x15a6cf813ab0003, quorum=localhost:49322, baseZNode=/hbase Transitioned node 3c1c23619d058b12671d9681e36bf99c from RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENED 2017-02-23 21:54:48,518 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:56195-0x15a6cf813ab0001, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/3c1c23619d058b12671d9681e36bf99c 2017-02-23 21:54:48,518 DEBUG [RS_OPEN_PRIORITY_REGION-xxx:54602-1] coordination.ZkOpenRegionCoordination(91): Transitioned 3c1c23619d058b12671d9681e36bf99c to OPENED in zk on xxx,54602,1487886883378 2017-02-23 21:54:48,519 DEBUG [RS_OPEN_PRIORITY_REGION-xxx:54602-1] handler.OpenRegionHandler(170): Opened hbase:backup,,1487886887259.3c1c23619d058b12671d9681e36bf99c. on xxx,54602,1487886883378 2017-02-23 21:54:48,519 DEBUG [AM.ZK.Worker-pool15-t2] master.AssignmentManager(960): Handling RS_ZK_REGION_OPENED, server=xxx,54602,1487886883378, region=3c1c23619d058b12671d9681e36bf99c, current_state={3c1c23619d058b12671d9681e36bf99c state=OPENING, ts=1487886888458, server=xxx,54602,1487886883378} 2017-02-23 21:54:48,520 INFO [AM.ZK.Worker-pool15-t2] master.RegionStates(1158): Transition {3c1c23619d058b12671d9681e36bf99c state=OPENING, ts=1487886888458, server=xxx,54602,1487886883378} to {3c1c23619d058b12671d9681e36bf99c state=OPEN, ts=1487886888520, server=xxx,54602,1487886883378} 2017-02-23 21:54:48,520 WARN [xxx:56195.activeMasterManager] master.RegionStates(349): Tried to create a state for a region already in RegionStates, used existing: {3c1c23619d058b12671d9681e36bf99c state=OPEN, ts=1487886888520, server=xxx,54602,1487886883378}, ignored new: OFFLINE 2017-02-23 21:54:48,520 DEBUG [AM.ZK.Worker-pool15-t2] coordination.ZkOpenRegionCoordination(350): Handling OPENED of 3c1c23619d058b12671d9681e36bf99c from xxx,56195,1487886883050; deleting unassigned node 2017-02-23 21:54:48,521 FATAL [xxx:56195.activeMasterManager] master.HMaster$2(1931): Failed to become active master java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:333) at java.util.concurrent.ConcurrentHashMap.containsKey(ConcurrentHashMap.java:1016) at java.util.concurrent.ConcurrentHashMap$KeySet.contains(ConcurrentHashMap.java:1437) at java.util.Collections$UnmodifiableCollection.contains(Collections.java:1057) at org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:3047) at org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:492) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:839) at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:222) at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1927) at java.lang.Thread.run(Thread.java:722) 2017-02-23 21:54:48,521 DEBUG [AM.ZK.Worker-pool15-t2] zookeeper.ZKAssign(479): master:56195-0x15a6cf813ab0001, quorum=localhost:49322, baseZNode=/hbase Deleted unassigned node 3c1c23619d058b12671d9681e36bf99c in expected state RS_ZK_REGION_OPENED 2017-02-23 21:54:48,521 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:56195-0x15a6cf813ab0001, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/region-in-transition/3c1c23619d058b12671d9681e36bf99c 2017-02-23 21:54:48,523 FATAL [xxx:56195.activeMasterManager] master.HMaster(2559): Master server abort: loaded coprocessors are: [org.apache.hadoop.hbase.backup.master.BackupController, org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint] 2017-02-23 21:54:48,523 FATAL [xxx:56195.activeMasterManager] master.HMaster(2562): Unhandled exception. Starting shutdown. java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:333) at java.util.concurrent.ConcurrentHashMap.containsKey(ConcurrentHashMap.java:1016) at java.util.concurrent.ConcurrentHashMap$KeySet.contains(ConcurrentHashMap.java:1437) at java.util.Collections$UnmodifiableCollection.contains(Collections.java:1057) at org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:3047) at org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:492) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:839) at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:222) at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1927) at java.lang.Thread.run(Thread.java:722) 2017-02-23 21:54:48,523 DEBUG [AM.ZK.Worker-pool15-t3] master.AssignmentManager$4(1358): Znode hbase:backup,,1487886887259.3c1c23619d058b12671d9681e36bf99c. deleted, state: {3c1c23619d058b12671d9681e36bf99c state=OPEN, ts=1487886888520, server=xxx,54602,1487886883378} 2017-02-23 21:54:48,524 DEBUG [AM.ZK.Worker-pool15-t3] master.RegionStates(505): Onlined 3c1c23619d058b12671d9681e36bf99c on xxx,54602,1487886883378 {ENCODED => 3c1c23619d058b12671d9681e36bf99c, NAME => 'hbase:backup,,1487886887259.3c1c23619d058b12671d9681e36bf99c.', STARTKEY => '', ENDKEY => ''} 2017-02-23 21:54:48,524 INFO [xxx:56195.activeMasterManager] regionserver.HRegionServer(1991): STOPPED: Unhandled exception. Starting shutdown. 2017-02-23 21:54:48,524 INFO [M:1;xxx:56195] procedure2.ProcedureExecutor(481): Stopping the procedure executor 2017-02-23 21:54:48,525 INFO [M:1;xxx:56195] wal.WALProcedureStore(239): Stopping the WAL Procedure Store 2017-02-23 21:54:48,539 DEBUG [ProcedureExecutorThread-0] lock.ZKInterProcessLockBase(328): Released /hbase/table-lock/hbase:backup/write-master:561950000000001 2017-02-23 21:54:48,965 INFO [M:1;xxx:56195] regionserver.HRegionServer(1066): stopping server xxx,56195,1487886883050 2017-02-23 21:54:48,965 DEBUG [M:1;xxx:56195] zookeeper.MetaTableLocator(627): Stopping MetaTableLocator 2017-02-23 21:54:48,966 INFO [M:1;xxx:56195] client.ConnectionManager$HConnectionImplementation(1687): Closing zookeeper sessionid=0x15a6cf813ab0005 2017-02-23 21:54:48,968 DEBUG [M:1;xxx:56195] ipc.RpcClientImpl(1115): Stopping rpc client 2017-02-23 21:54:48,968 DEBUG [RpcServer.reader=7,bindAddress=xxx,port=54602] ipc.RpcServer$Listener(882): RpcServer.listener,port=54602: DISCONNECTING client xxx:60464 because read count=-1. Number of active connections: 4 2017-02-23 21:54:48,968 DEBUG [RpcServer.reader=5,bindAddress=xxx,port=54602] ipc.RpcServer$Listener(882): RpcServer.listener,port=54602: DISCONNECTING client xxx:48944 because read count=-1. Number of active connections: 4 2017-02-23 21:54:48,978 INFO [M:1;xxx:56195] regionserver.HRegionServer(1094): stopping server xxx,56195,1487886883050; all regions closed. 2017-02-23 21:54:48,978 INFO [M:1;xxx:56195] hbase.ChoreService(302): Chore service for: xxx,56195,1487886883050 had [[ScheduledChore: Name: LogsCleaner Period: 60000 Unit: MILLISECONDS], [ScheduledChore: Name: HFileCleaner Period: 60000 Unit: MILLISECONDS]] on shutdown 2017-02-23 21:54:48,979 DEBUG [M:1;xxx:56195] master.HMaster(1243): Stopping service threads 2017-02-23 21:54:48,981 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): regionserver:54602-0x15a6cf813ab0003, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2017-02-23 21:54:48,981 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:56195-0x15a6cf813ab0001, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2017-02-23 21:54:48,981 INFO [M:1;xxx:56195] client.ConnectionManager$HConnectionImplementation(1687): Closing zookeeper sessionid=0x15a6cf813ab000c 2017-02-23 21:54:48,981 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2017-02-23 21:54:48,981 DEBUG [Time-limited test-EventThread] zookeeper.ZKUtil(369): regionserver:54602-0x15a6cf813ab0003, quorum=localhost:49322, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2017-02-23 21:54:48,981 DEBUG [Time-limited test-EventThread] zookeeper.ZKUtil(369): master:56195-0x15a6cf813ab0001, quorum=localhost:49322, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2017-02-23 21:54:48,982 DEBUG [M:1;xxx:56195] ipc.RpcClientImpl(1115): Stopping rpc client 2017-02-23 21:54:48,982 DEBUG [Time-limited test-EventThread] zookeeper.ZKUtil(369): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2017-02-23 21:54:48,983 DEBUG [RpcServer.reader=6,bindAddress=xxx,port=54602] ipc.RpcServer$Listener(882): RpcServer.listener,port=54602: DISCONNECTING client xxx:60007 because read count=-1. Number of active connections: 2 2017-02-23 21:54:48,983 DEBUG [Time-limited test-EventThread] zookeeper.ZKUtil(369): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2017-02-23 21:54:48,983 DEBUG [Time-limited test-EventThread] master.ActiveMasterManager(130): No master available. Notifying waiting threads 2017-02-23 21:54:48,984 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2017-02-23 21:54:48,984 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:56195-0x15a6cf813ab0001, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2017-02-23 21:54:48,985 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): regionserver:54602-0x15a6cf813ab0003, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2017-02-23 21:54:48,985 DEBUG [xxx:55048.activeMasterManager] zookeeper.ZKUtil(367): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2017-02-23 21:54:48,985 DEBUG [Time-limited test-EventThread] zookeeper.ZKUtil(367): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2017-02-23 21:54:48,985 DEBUG [Time-limited test-EventThread] master.ActiveMasterManager(126): A master is now available 2017-02-23 21:54:48,986 INFO [xxx:55048.activeMasterManager] master.ActiveMasterManager(170): Deleting ZNode for /hbase/backup-masters/xxx,55048,1487886883101 from backup master directory 2017-02-23 21:54:48,988 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/xxx,55048,1487886883101 2017-02-23 21:54:48,988 WARN [xxx:55048.activeMasterManager] hbase.ZNodeClearer(58): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2017-02-23 21:54:48,989 INFO [xxx:55048.activeMasterManager] master.ActiveMasterManager(179): Registered Active Master=xxx,55048,1487886883101 2017-02-23 21:54:48,993 INFO [M:1;xxx:56195] hbase.ChoreService(302): Chore service for: xxx,56195,1487886883050_splitLogManager_ had [[ScheduledChore: Name: SplitLogManager Timeout Monitor Period: 1000 Unit: MILLISECONDS]] on shutdown 2017-02-23 21:54:48,993 INFO [M:1;xxx:56195] master.LogRollMasterProcedureManager(55): stop: server shutting down. 2017-02-23 21:54:48,993 INFO [M:1;xxx:56195] flush.MasterFlushTableProcedureManager(79): stop: server shutting down. 2017-02-23 21:54:48,993 INFO [M:1;xxx:56195] ipc.RpcServer(2248): Stopping server on 56195 2017-02-23 21:54:48,994 INFO [RpcServer.listener,port=56195] ipc.RpcServer$Listener(786): RpcServer.listener,port=56195: stopping 2017-02-23 21:54:48,994 INFO [RpcServer.responder] ipc.RpcServer$Responder(1028): RpcServer.responder: stopped 2017-02-23 21:54:48,995 INFO [M:1;xxx:56195] zookeeper.RecoverableZooKeeper(188): Node /hbase/rs/xxx,56195,1487886883050 already deleted, retry=false 2017-02-23 21:54:48,996 INFO [RpcServer.responder] ipc.RpcServer$Responder(931): RpcServer.responder: stopping 2017-02-23 21:54:48,997 INFO [M:1;xxx:56195] regionserver.HRegionServer(1138): stopping server xxx,56195,1487886883050; zookeeper connection closed. 2017-02-23 21:54:48,997 INFO [M:1;xxx:56195] regionserver.HRegionServer(1141): M:1;xxx:56195 exiting 2017-02-23 21:54:49,013 DEBUG [xxx:55048.activeMasterManager] util.FSTableDescriptors(731): Current tableInfoPath = hdfs://localhost:59749/user/nobody/test-data/3c09ad53-b369-4742-aa28-6ea8cf270e8b/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2017-02-23 21:54:49,019 DEBUG [xxx:55048.activeMasterManager] util.FSTableDescriptors(735): TableInfo already exists.. Skipping creation 2017-02-23 21:54:49,029 INFO [xxx:55048.activeMasterManager] fs.HFileSystem(256): Added intercepting call to namenode#getBlockLocations so can do block reordering using class class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2017-02-23 21:54:49,029 DEBUG [xxx:55048.activeMasterManager] coordination.ZKSplitLogManagerCoordination(886): Distributed log replay=false 2017-02-23 21:54:49,031 INFO [xxx:55048.activeMasterManager] coordination.ZKSplitLogManagerCoordination(599): Found 0 orphan tasks and 0 rescan nodes 2017-02-23 21:54:49,031 DEBUG [xxx:55048.activeMasterManager] util.FSTableDescriptors(208): Fetching table descriptors from the filesystem. 2017-02-23 21:54:49,055 INFO [xxx:55048.activeMasterManager] zookeeper.RecoverableZooKeeper(120): Process identifier=hconnection-0x10bc1446 connecting to ZooKeeper ensemble=localhost:49322 2017-02-23 21:54:49,055 INFO [M:2;xxx:55048] regionserver.HRegionServer(807): ClusterId : 33c12452-369c-4de8-8dd0-16f76986b10f 2017-02-23 21:54:49,057 DEBUG [xxx:55048.activeMasterManager-EventThread] zookeeper.ZooKeeperWatcher(515): hconnection-0x10bc14460x0, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2017-02-23 21:54:49,057 DEBUG [xxx:55048.activeMasterManager-EventThread] zookeeper.ZooKeeperWatcher(578): hconnection-0x10bc1446-0x15a6cf813ab000e connected 2017-02-23 21:54:49,058 DEBUG [xxx:55048.activeMasterManager] ipc.AbstractRpcClient(105): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@479dee25, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2017-02-23 21:54:49,060 INFO [xxx:55048.activeMasterManager] balancer.StochasticLoadBalancer(155): loading config 2017-02-23 21:54:49,061 DEBUG [xxx:55048.activeMasterManager] zookeeper.ZKUtil(369): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2017-02-23 21:54:49,062 DEBUG [xxx:55048.activeMasterManager] zookeeper.ZKUtil(369): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2017-02-23 21:54:49,063 DEBUG [xxx:55048.activeMasterManager] zookeeper.ZKUtil(369): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2017-02-23 21:54:49,064 DEBUG [xxx:55048.activeMasterManager] zookeeper.ZKUtil(369): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2017-02-23 21:54:49,068 DEBUG [xxx:55048.activeMasterManager] zookeeper.ZKUtil(367): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/xxx,54602,1487886883378 2017-02-23 21:54:49,068 DEBUG [xxx:55048.activeMasterManager] zookeeper.RegionServerTracker(93): Added tracking of RS /hbase/rs/xxx,54602,1487886883378 2017-02-23 21:54:49,069 INFO [xxx:55048.activeMasterManager] master.HMaster(673): Server active/primary master=xxx,55048,1487886883101, sessionid=0x15a6cf813ab0002, setting cluster-up flag (Was=true) 2017-02-23 21:54:49,069 INFO [xxx:55048.activeMasterManager] procedure.ProcedureManagerHost(71): User procedure org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager was loaded successfully. 2017-02-23 21:54:49,072 DEBUG [xxx:55048.activeMasterManager] zookeeper.RecoverableZooKeeper(584): Node /hbase/online-snapshot/acquired already exists 2017-02-23 21:54:49,073 INFO [xxx:55048.activeMasterManager] procedure.ZKProcedureUtil(270): Clearing all procedure znodes: /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2017-02-23 21:54:49,075 DEBUG [xxx:55048.activeMasterManager] procedure.ZKProcedureCoordinatorRpcs(238): Starting the controller for procedure member:xxx,55048,1487886883101 2017-02-23 21:54:49,077 DEBUG [xxx:55048.activeMasterManager] zookeeper.RecoverableZooKeeper(584): Node /hbase/rolllog-proc/acquired already exists 2017-02-23 21:54:49,078 INFO [xxx:55048.activeMasterManager] procedure.ZKProcedureUtil(270): Clearing all procedure znodes: /hbase/rolllog-proc/acquired /hbase/rolllog-proc/reached /hbase/rolllog-proc/abort 2017-02-23 21:54:49,079 DEBUG [xxx:55048.activeMasterManager] procedure.ZKProcedureCoordinatorRpcs(238): Starting the controller for procedure member:xxx,55048,1487886883101 2017-02-23 21:54:49,080 DEBUG [xxx:55048.activeMasterManager] zookeeper.RecoverableZooKeeper(584): Node /hbase/flush-table-proc/acquired already exists 2017-02-23 21:54:49,081 INFO [xxx:55048.activeMasterManager] procedure.ZKProcedureUtil(270): Clearing all procedure znodes: /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2017-02-23 21:54:49,082 DEBUG [xxx:55048.activeMasterManager] procedure.ZKProcedureCoordinatorRpcs(238): Starting the controller for procedure member:xxx,55048,1487886883101 2017-02-23 21:54:49,082 INFO [xxx:55048.activeMasterManager] master.MasterCoprocessorHost(92): System coprocessor loading is enabled 2017-02-23 21:54:49,083 INFO [xxx:55048.activeMasterManager] coprocessor.CoprocessorHost(160): System coprocessor org.apache.hadoop.hbase.backup.master.BackupController was loaded successfully with priority (536870911). 2017-02-23 21:54:49,083 DEBUG [xxx:55048.activeMasterManager] executor.ExecutorService(100): Starting executor service name=MASTER_OPEN_REGION-xxx:55048, corePoolSize=5, maxPoolSize=5 2017-02-23 21:54:49,083 DEBUG [xxx:55048.activeMasterManager] executor.ExecutorService(100): Starting executor service name=MASTER_CLOSE_REGION-xxx:55048, corePoolSize=5, maxPoolSize=5 2017-02-23 21:54:49,083 DEBUG [xxx:55048.activeMasterManager] executor.ExecutorService(100): Starting executor service name=MASTER_SERVER_OPERATIONS-xxx:55048, corePoolSize=5, maxPoolSize=5 2017-02-23 21:54:49,083 DEBUG [xxx:55048.activeMasterManager] executor.ExecutorService(100): Starting executor service name=MASTER_META_SERVER_OPERATIONS-xxx:55048, corePoolSize=5, maxPoolSize=5 2017-02-23 21:54:49,083 DEBUG [xxx:55048.activeMasterManager] executor.ExecutorService(100): Starting executor service name=M_LOG_REPLAY_OPS-xxx:55048, corePoolSize=10, maxPoolSize=10 2017-02-23 21:54:49,084 DEBUG [xxx:55048.activeMasterManager] executor.ExecutorService(100): Starting executor service name=MASTER_TABLE_OPERATIONS-xxx:55048, corePoolSize=1, maxPoolSize=1 2017-02-23 21:54:49,084 INFO [xxx:55048.activeMasterManager] procedure2.ProcedureExecutor(436): Starting procedure executor threads=33 2017-02-23 21:54:49,084 INFO [xxx:55048.activeMasterManager] wal.WALProcedureStore(318): Starting WAL Procedure Store lease recovery 2017-02-23 21:54:49,086 INFO [xxx:55048.activeMasterManager] util.FSHDFSUtils(194): Recovering lease on dfs file hdfs://localhost:59749/user/nobody/test-data/3c09ad53-b369-4742-aa28-6ea8cf270e8b/MasterProcWALs/state-00000000000000000002.log 2017-02-23 21:54:49,087 INFO [xxx:55048.activeMasterManager] util.FSHDFSUtils(283): recoverLease=true, attempt=0 on file=hdfs://localhost:59749/user/nobody/test-data/3c09ad53-b369-4742-aa28-6ea8cf270e8b/MasterProcWALs/state-00000000000000000002.log after 1ms 2017-02-23 21:54:49,087 DEBUG [xxx:55048.activeMasterManager] wal.WALProcedureStore(1045): Opening state-log: FileStatus{path=hdfs://localhost:59749/user/nobody/test-data/3c09ad53-b369-4742-aa28-6ea8cf270e8b/MasterProcWALs/state-00000000000000000002.log; isDirectory=false; length=2600; replication=1; blocksize=134217728; modification_time=1487886888963; access_time=1487886887535; owner=nobody; group=supergroup; permission=rw-r--r--; isSymlink=false} 2017-02-23 21:54:49,097 DEBUG [xxx:55048.activeMasterManager] wal.WALProcedureStore(861): Roll new state log: 3 2017-02-23 21:54:49,099 INFO [xxx:55048.activeMasterManager] wal.WALProcedureStore(347): Lease acquired for flushLogId: 3 2017-02-23 21:54:49,103 DEBUG [xxx:55048.activeMasterManager] procedure2.ProcedureExecutor(311): Loading procedure state=FINISHED isFailed=false: CreateTableProcedure (table=hbase:namespace) id=1 owner=nobody state=FINISHED 2017-02-23 21:54:49,103 DEBUG [xxx:55048.activeMasterManager] procedure2.ProcedureExecutor(311): Loading procedure state=RUNNABLE isFailed=false: CreateTableProcedure (table=hbase:backup) id=2 owner=nobody state=RUNNABLE 2017-02-23 21:54:49,103 DEBUG [xxx:55048.activeMasterManager] procedure2.ProcedureExecutor(343): The procedure is completed state=FINISHED isFailed=false: CreateTableProcedure (table=hbase:namespace) id=1 owner=nobody state=FINISHED 2017-02-23 21:54:49,106 DEBUG [xxx:55048.activeMasterManager] cleaner.CleanerChore(91): initialize cleaner=org.apache.hadoop.hbase.backup.master.BackupLogCleaner 2017-02-23 21:54:49,106 DEBUG [xxx:55048.activeMasterManager] cleaner.CleanerChore(91): initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2017-02-23 21:54:49,106 INFO [xxx:55048.activeMasterManager] zookeeper.RecoverableZooKeeper(120): Process identifier=replicationLogCleaner connecting to ZooKeeper ensemble=localhost:49322 2017-02-23 21:54:49,107 DEBUG [ProcedureExecutorThread-0] lock.ZKInterProcessLockBase(226): Acquired a lock for /hbase/table-lock/hbase:backup/write-master:550480000000002 2017-02-23 21:54:49,109 DEBUG [xxx:55048.activeMasterManager-EventThread] zookeeper.ZooKeeperWatcher(515): replicationLogCleaner0x0, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2017-02-23 21:54:49,109 DEBUG [xxx:55048.activeMasterManager-EventThread] zookeeper.ZooKeeperWatcher(578): replicationLogCleaner-0x15a6cf813ab000f connected 2017-02-23 21:54:49,109 DEBUG [xxx:55048.activeMasterManager] zookeeper.RecoverableZooKeeper(584): Node /hbase/replication/rs already exists 2017-02-23 21:54:49,110 DEBUG [xxx:55048.activeMasterManager] cleaner.CleanerChore(91): initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2017-02-23 21:54:49,110 DEBUG [xxx:55048.activeMasterManager] cleaner.CleanerChore(91): initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotLogCleaner 2017-02-23 21:54:49,111 DEBUG [xxx:55048.activeMasterManager] cleaner.CleanerChore(91): initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2017-02-23 21:54:49,111 DEBUG [xxx:55048.activeMasterManager] cleaner.CleanerChore(91): initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2017-02-23 21:54:49,111 DEBUG [xxx:55048.activeMasterManager] cleaner.CleanerChore(91): initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2017-02-23 21:54:49,111 INFO [xxx:55048.activeMasterManager] master.ServerManager(1037): Waiting for region servers count to settle; currently checked in 0, slept for 0 ms, expecting minimum of 1, maximum of 1, timeout of 30000 ms, interval of 1500 ms. 2017-02-23 21:54:49,157 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,207 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,258 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,308 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,358 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,409 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,453 INFO [org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@4658a66c] blockmanagement.BlockManager(1594): BLOCK* neededReplications = 0, pendingReplications = 0. 2017-02-23 21:54:49,459 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,510 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,560 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,611 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,661 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,712 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,762 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,813 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,863 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,914 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:49,964 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:50,015 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:50,065 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:50,116 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:50,166 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:50,217 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:50,267 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:50,318 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:50,368 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:50,419 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:50,428 DEBUG [RS:0;xxx:54602] ipc.RpcClientImpl$Connection(345): Use SIMPLE authentication for service RegionServerStatusService, sasl=false 2017-02-23 21:54:50,428 DEBUG [RS:0;xxx:54602] ipc.RpcClientImpl$Connection(710): Connecting to xxx/xxx:55048 2017-02-23 21:54:50,429 DEBUG [RpcServer.listener,port=55048] ipc.RpcServer$Listener(850): RpcServer.listener,port=55048: connection from xxx:51789; # active connections: 1 2017-02-23 21:54:50,431 INFO [RpcServer.reader=1,bindAddress=xxx,port=55048] ipc.RpcServer$Connection(1683): Connection from xxx port: 51789 with version info: version: "1.1.2.2.6.0.0-SNAPSHOT" url: "git://xxx/grid/0/nobody/workspace/build-support/SOURCES/hbase" revision: "87b4908be2ff0ee4c12025175a32057af7018bae" user: "nobody" date: "Thu Feb 23 20:49:04 UTC 2017" src_checksum: "dcba0c744a09baccb7e88330156a1cf6" 2017-02-23 21:54:50,432 INFO [RpcServer.FifoWFPBQ.priority.handler=19,queue=1,port=55048] master.ServerManager(452): Registering server=xxx,54602,1487886883378 2017-02-23 21:54:50,464 INFO [xxx:55048.activeMasterManager] master.ServerManager(1054): Finished waiting for region servers count to settle; checked in 1, slept for 1353 ms, expecting minimum of 1, maximum of 1, master is running 2017-02-23 21:54:50,469 DEBUG [ProcedureExecutorThread-0] procedure.ProcedureSyncWait(120): Waiting on server to assign region(s) 2017-02-23 21:54:50,470 INFO [ProcedureExecutorThread-0] zookeeper.ZKTableStateManager(100): Moving table hbase:backup state from ENABLED to ENABLING 2017-02-23 21:54:50,471 INFO [xxx:55048.activeMasterManager] master.MasterFileSystem(290): Log folder hdfs://localhost:59749/user/nobody/test-data/3c09ad53-b369-4742-aa28-6ea8cf270e8b/WALs/xxx,54602,1487886883378 belongs to an existing region server 2017-02-23 21:54:50,472 INFO [ProcedureExecutorThread-0] master.AssignmentManager(1627): Assigning 1 region(s) to xxx,54602,1487886883378 2017-02-23 21:54:50,472 DEBUG [ProcedureExecutorThread-0] zookeeper.ZKAssign(174): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Async create of unassigned node 3c1c23619d058b12671d9681e36bf99c with OFFLINE state 2017-02-23 21:54:50,474 DEBUG [Time-limited test-EventThread] master.OfflineCallback(69): rs={3c1c23619d058b12671d9681e36bf99c state=OFFLINE, ts=1487886890472, server=null}, server=xxx,54602,1487886883378 2017-02-23 21:54:50,475 DEBUG [Time-limited test-EventThread] master.OfflineCallback$ExistCallback(106): rs={3c1c23619d058b12671d9681e36bf99c state=OFFLINE, ts=1487886890472, server=null}, server=xxx,54602,1487886883378 2017-02-23 21:54:50,478 DEBUG [ProcedureExecutorThread-0] master.AssignmentManager(1680): xxx,54602,1487886883378 unassigned znodes=1 of total=1; oldCounter=0 2017-02-23 21:54:50,478 INFO [ProcedureExecutorThread-0] master.RegionStates(1158): Transition {3c1c23619d058b12671d9681e36bf99c state=OFFLINE, ts=1487886890472, server=null} to {3c1c23619d058b12671d9681e36bf99c state=PENDING_OPEN, ts=1487886890478, server=xxx,54602,1487886883378} 2017-02-23 21:54:50,478 DEBUG [ProcedureExecutorThread-0] master.ServerManager(963): New admin connection to xxx,54602,1487886883378 2017-02-23 21:54:50,479 DEBUG [ProcedureExecutorThread-0] ipc.RpcClientImpl$Connection(345): Use SIMPLE authentication for service AdminService, sasl=false 2017-02-23 21:54:50,479 DEBUG [ProcedureExecutorThread-0] ipc.RpcClientImpl$Connection(710): Connecting to xxx/xxx:54602 2017-02-23 21:54:50,480 DEBUG [RpcServer.listener,port=54602] ipc.RpcServer$Listener(850): RpcServer.listener,port=54602: connection from xxx:57322; # active connections: 2 2017-02-23 21:54:50,481 INFO [RpcServer.reader=8,bindAddress=xxx,port=54602] ipc.RpcServer$Connection(1683): Connection from xxx port: 57322 with version info: version: "1.1.2.2.6.0.0-SNAPSHOT" url: "git://xxx/grid/0/nobody/workspace/build-support/SOURCES/hbase" revision: "87b4908be2ff0ee4c12025175a32057af7018bae" user: "nobody" date: "Thu Feb 23 20:49:04 UTC 2017" src_checksum: "dcba0c744a09baccb7e88330156a1cf6" 2017-02-23 21:54:50,482 DEBUG [xxx:55048.activeMasterManager] ipc.RpcClientImpl$Connection(345): Use SIMPLE authentication for service AdminService, sasl=false 2017-02-23 21:54:50,482 DEBUG [xxx:55048.activeMasterManager] ipc.RpcClientImpl$Connection(710): Connecting to xxx/xxx:54602 2017-02-23 21:54:50,483 DEBUG [RpcServer.listener,port=54602] ipc.RpcServer$Listener(850): RpcServer.listener,port=54602: connection from xxx:57355; # active connections: 3 2017-02-23 21:54:50,484 INFO [RpcServer.reader=9,bindAddress=xxx,port=54602] ipc.RpcServer$Connection(1683): Connection from xxx port: 57355 with version info: version: "1.1.2.2.6.0.0-SNAPSHOT" url: "git://xxx/grid/0/nobody/workspace/build-support/SOURCES/hbase" revision: "87b4908be2ff0ee4c12025175a32057af7018bae" user: "nobody" date: "Thu Feb 23 20:49:04 UTC 2017" src_checksum: "dcba0c744a09baccb7e88330156a1cf6" 2017-02-23 21:54:50,485 INFO [xxx:55048.activeMasterManager] master.RegionStates(1158): Transition {1588230740 state=OFFLINE, ts=1487886890477, server=null} to {1588230740 state=OPEN, ts=1487886890485, server=xxx,54602,1487886883378} 2017-02-23 21:54:50,485 DEBUG [xxx:55048.activeMasterManager] master.RegionStates(505): Onlined 1588230740 on xxx,54602,1487886883378 {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2017-02-23 21:54:50,486 INFO [xxx:55048.activeMasterManager] master.ServerManager(684): AssignmentManager hasn't finished failover cleanup; waiting 2017-02-23 21:54:50,487 INFO [xxx:55048.activeMasterManager] master.HMaster(1054): hbase:meta with replicaId 0 assigned=0, rit=false, location=xxx,54602,1487886883378 2017-02-23 21:54:50,490 DEBUG [hconnection-0x2e6d1362-shared--pool19-t1] ipc.RpcClientImpl$Connection(345): Use SIMPLE authentication for service ClientService, sasl=false 2017-02-23 21:54:50,491 DEBUG [hconnection-0x2e6d1362-shared--pool19-t1] ipc.RpcClientImpl$Connection(710): Connecting to xxx/xxx:54602 2017-02-23 21:54:50,491 DEBUG [RpcServer.listener,port=54602] ipc.RpcServer$Listener(850): RpcServer.listener,port=54602: connection from xxx:57469; # active connections: 4 2017-02-23 21:54:50,492 INFO [RpcServer.reader=0,bindAddress=xxx,port=54602] ipc.RpcServer$Connection(1683): Connection from xxx port: 57469 with version info: version: "1.1.2.2.6.0.0-SNAPSHOT" url: "git://xxx/grid/0/nobody/workspace/build-support/SOURCES/hbase" revision: "87b4908be2ff0ee4c12025175a32057af7018bae" user: "nobody" date: "Thu Feb 23 20:49:04 UTC 2017" src_checksum: "dcba0c744a09baccb7e88330156a1cf6" 2017-02-23 21:54:50,493 WARN [RpcServer.FifoWFPBQ.priority.handler=17,queue=1,port=54602] regionserver.RSRpcServices(1505): Attempted open of 3c1c23619d058b12671d9681e36bf99c but already online on this server 2017-02-23 21:54:50,493 DEBUG [ProcedureExecutorThread-0] master.AssignmentManager(2320): ALREADY_OPENED hbase:backup,,1487886887259.3c1c23619d058b12671d9681e36bf99c. to xxx,54602,1487886883378 2017-02-23 21:54:50,498 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/region-in-transition/3c1c23619d058b12671d9681e36bf99c 2017-02-23 21:54:50,498 DEBUG [ProcedureExecutorThread-0] zookeeper.ZKAssign(479): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Deleted unassigned node 3c1c23619d058b12671d9681e36bf99c in expected state M_ZK_REGION_OFFLINE 2017-02-23 21:54:50,499 INFO [ProcedureExecutorThread-0] master.RegionStates(1158): Transition {3c1c23619d058b12671d9681e36bf99c state=PENDING_OPEN, ts=1487886890478, server=xxx,54602,1487886883378} to {3c1c23619d058b12671d9681e36bf99c state=OPEN, ts=1487886890499, server=xxx,54602,1487886883378} 2017-02-23 21:54:50,499 DEBUG [ProcedureExecutorThread-0] master.RegionStates(505): Onlined 3c1c23619d058b12671d9681e36bf99c on xxx,54602,1487886883378 {ENCODED => 3c1c23619d058b12671d9681e36bf99c, NAME => 'hbase:backup,,1487886887259.3c1c23619d058b12671d9681e36bf99c.', STARTKEY => '', ENDKEY => ''} 2017-02-23 21:54:50,499 DEBUG [ProcedureExecutorThread-0] master.AssignmentManager(1822): Bulk assigning done for xxx,54602,1487886883378 2017-02-23 21:54:50,500 INFO [ProcedureExecutorThread-0] zookeeper.ZKTableStateManager(100): Moving table hbase:backup state from ENABLING to ENABLED 2017-02-23 21:54:50,500 INFO [xxx:55048.activeMasterManager] hbase.MetaMigrationConvertingToPB(131): META already up-to date with PB serialization 2017-02-23 21:54:50,513 WARN [xxx:55048.activeMasterManager] master.RegionStates(349): Tried to create a state for a region already in RegionStates, used existing: {3c1c23619d058b12671d9681e36bf99c state=OPEN, ts=1487886890499, server=xxx,54602,1487886883378}, ignored new: OPEN 2017-02-23 21:54:50,516 DEBUG [xxx:55048.activeMasterManager] master.AssignmentManager(546): Found {ENCODED => 3c1c23619d058b12671d9681e36bf99c, NAME => 'hbase:backup,,1487886887259.3c1c23619d058b12671d9681e36bf99c.', STARTKEY => '', ENDKEY => ''} out on cluster 2017-02-23 21:54:50,516 INFO [xxx:55048.activeMasterManager] master.AssignmentManager(629): Found regions out on cluster or in RIT; presuming failover 2017-02-23 21:54:50,522 INFO [xxx:55048.activeMasterManager] master.AssignmentManager(505): Joined the cluster in 22ms, failover=true 2017-02-23 21:54:50,530 DEBUG [xxx:55048.activeMasterManager] zookeeper.ZKUtil(367): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Set watcher on existing znode=/hbase/namespace 2017-02-23 21:54:50,532 DEBUG [xxx:55048.activeMasterManager] hbase.ZKNamespaceManager(196): Updating namespace cache from node default with data: \x0A\x07default 2017-02-23 21:54:50,533 DEBUG [xxx:55048.activeMasterManager] hbase.ZKNamespaceManager(196): Updating namespace cache from node hbase with data: \x0A\x05hbase 2017-02-23 21:54:50,550 DEBUG [xxx:55048.activeMasterManager] zookeeper.RecoverableZooKeeper(584): Node /hbase/namespace/default already exists 2017-02-23 21:54:50,551 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2017-02-23 21:54:50,552 DEBUG [xxx:55048.activeMasterManager] zookeeper.RecoverableZooKeeper(584): Node /hbase/namespace/hbase already exists 2017-02-23 21:54:50,554 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2017-02-23 21:54:50,554 INFO [xxx:55048.activeMasterManager] master.HMaster(868): Master has completed initialization 2017-02-23 21:54:50,554 INFO [xxx:55048.activeMasterManager] quotas.MasterQuotaManager(74): Quota support disabled 2017-02-23 21:54:50,578 INFO [xxx:55048.activeMasterManager] master.HMaster$1(1661): Client=null/null create 'hbase:backup', {NAME => 'meta', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL => 'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME => 'session', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL => 'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'} 2017-02-23 21:54:50,613 DEBUG [xxx:55048.activeMasterManager] procedure2.ProcedureExecutor(701): Procedure CreateTableProcedure (table=hbase:backup) id=3 owner=nobody state=RUNNABLE added to the store. 2017-02-23 21:54:50,632 DEBUG [Time-limited test-EventThread] zookeeper.ZooKeeperWatcher(515): master:55048-0x15a6cf813ab0002, quorum=localhost:49322, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2017-02-23 21:54:50,633 INFO [Time-limited test] master.MasterRpcServices(281): Client=null/null set balanceSwitch=false 2017-02-23 21:54:50,633 INFO [Time-limited test] hbase.MiniHBaseCluster(451): Stopping Thread[M:1;xxx:56195,5,]