Thu Nov 4 11:34:07 CST 2021 Starting master on C3HM1 core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 512460 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1048576 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) unlimited virtual memory (kbytes, -v) unlimited file locks (-x) unlimited 2021-11-04 11:34:07,513 INFO [main] util.VersionInfo: HBase 1.7.1 2021-11-04 11:34:07,513 INFO [main] util.VersionInfo: Source code repository git://localhost.localdomain/home/bharathv/IdeaProjects/rc/hbase-rc-clone revision=2d9273667e418e7023f9104a830cdcb8233b6f25 2021-11-04 11:34:07,513 INFO [main] util.VersionInfo: Compiled by bharathv on Fri Jul 16 00:20:26 PDT 2021 2021-11-04 11:34:07,513 INFO [main] util.VersionInfo: From source with checksum cb15e8fa7fadc65448aa9a30e3f40b27 2021-11-04 11:34:07,683 INFO [main] util.ServerCommandLine: env:PATH=/home/java/jdk1.8.0_271/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2021-11-04 11:34:07,683 INFO [main] util.ServerCommandLine: env:HBASE_LOGFILE=hbase-root-master-C3HM1.log 2021-11-04 11:34:07,683 INFO [main] util.ServerCommandLine: env:MSG_OPT=false 2021-11-04 11:34:07,683 INFO [main] util.ServerCommandLine: env:HBASE_ZOOKEEPER_OPTS= -Dfile.encoding=UTF8 -Duser.timezone=GMT+08 2021-11-04 11:34:07,683 INFO [main] util.ServerCommandLine: env:JAVA_HOME=/home/java/jdk1.8.0_271 2021-11-04 11:34:07,683 INFO [main] util.ServerCommandLine: env:TERM=xterm 2021-11-04 11:34:07,683 INFO [main] util.ServerCommandLine: env:JRE_HOME=/home/java/jdk1.8.0_271/jre 2021-11-04 11:34:07,683 INFO [main] util.ServerCommandLine: env:HBASE_SHELL_OPTS= -Dfile.encoding=UTF8 -Duser.timezone=GMT+08 2021-11-04 11:34:07,683 INFO [main] util.ServerCommandLine: env:JVM_PID=169 2021-11-04 11:34:07,683 INFO [main] util.ServerCommandLine: env:TIME_OPTS=-Dfile.encoding=UTF8 -Duser.timezone=GMT+08 2021-11-04 11:34:07,683 INFO [main] util.ServerCommandLine: env:HBASE_REST_OPTS= -Dfile.encoding=UTF8 -Duser.timezone=GMT+08 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:PWD=/home/evaluation/hbase-1.7.1 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:JRUBY_OPTS= 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:HBASE_ROOT_LOGGER=INFO,RFA 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:PHOS_OPTS= 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:HBASE_ENV_INIT=true 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:HBASE_IDENT_STRING=root 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:HBASE_ZNODE_FILE=/tmp/hbase-root-master.znode 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:HBASE_LOG_PREFIX=hbase-root-master-C3HM1 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:HBASE_LOG_DIR=/home/evaluation/hbase-1.7.1/bin/../logs 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:OLDPWD=/ 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:USER=root 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:HBASE_MASTER_OPTS= -Dfile.encoding=UTF8 -Duser.timezone=GMT+08 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: oyu/evaluation/hbase-1.7.1/bin/../lib/log4j-1.2.17.jar:/home/evaluation/hbase-1.7.1/bin/../lib/metrics-core-2.2.0.jar:/home/evaluation/hbase-1.7.1/bin/../lib/metrics-core-3.1.2.jar:/home/evaluation/hbase-1.7.1/bin/../lib/netty-all-4.1.8.Final.jar:/home/evaluation/hbase-1.7.1/bin/../lib/paranamer-2.3.jar:/home/evaluation/hbase-1.7.1/bin/../lib/protobuf-java-2.5.0.jar:/home/evaluation/hbase-1.7.1/bin/../lib/servlet-api-2.5-6.1.14.jar:/home/evaluation/hbase-1.7.1/bin/../lib/servlet-api-2.5.jar:/home/evaluation/hbase-1.7.1/bin/../lib/slf4j-api-1.7.25.jar:/home/evaluation/hbase-1.7.1/bin/../lib/slf4j-log4j12-1.7.25.jar:/home/evaluation/hbase-1.7.1/bin/../lib/snappy-java-1.0.5.jar:/home/evaluation/hbase-1.7.1/bin/../lib/spymemcached-2.11.6.jar:/home/evaluation/hbase-1.7.1/bin/../lib/xmlenc-0.52.jar:/home/evaluation/hbase-1.7.1/bin/../lib/xz-1.0.jar:/home/evaluation/hbase-1.7.1/bin/../lib/zookeeper-3.4.10.jar: 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:HBASE_MANAGES_ZK=false 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:HBASE_AUTOSTART_FILE=/tmp/hbase-root-master.autostart 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:HOSTNAME=C3HM1 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:HBASE_NICENESS=0 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:HBASE_SECURITY_LOGGER=INFO,RFAS 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:HBASE_HOME=/home/evaluation/hbase-1.7.1/bin/.. 2021-11-04 11:34:07,684 INFO [main] util.ServerCommandLine: env:HBASE_THRIFT_OPTS= -Dfile.encoding=UTF8 -Duser.timezone=GMT+08 2021-11-04 11:34:07,685 INFO [main] util.ServerCommandLine: eb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: 2021-11-04 11:34:07,685 INFO [main] util.ServerCommandLine: env:SHLVL=1 2021-11-04 11:34:07,685 INFO [main] util.ServerCommandLine: env:HOME=/root 2021-11-04 11:34:07,685 INFO [main] util.ServerCommandLine: env:MALLOC_ARENA_MAX=4 2021-11-04 11:34:07,686 INFO [main] util.ServerCommandLine: vmName=Java HotSpot(TM) 64-Bit Server VM, vmVendor=Oracle Corporation, vmVersion=25.271-b09 2021-11-04 11:34:07,686 INFO [main] util.ServerCommandLine: vmInputArguments=[-Dproc_master, -XX:OnOutOfMemoryError=kill -9 %p, -Dfile.encoding=UTF8, -Duser.timezone=GMT+08, -Dhbase.log.dir=/home/evaluation/hbase-1.7.1/bin/../logs, -Dhbase.log.file=hbase-root-master-C3HM1.log, -Dhbase.home.dir=/home/evaluation/hbase-1.7.1/bin/.., -Dhbase.id.str=root, -Dhbase.root.logger=INFO,RFA, -Dhbase.security.logger=INFO,RFAS] 2021-11-04 11:34:07,782 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2021-11-04 11:34:08,030 INFO [main] regionserver.RSRpcServices: master/C3HM1/172.27.0.2:16000 server-side HConnection retries=350 2021-11-04 11:34:08,083 INFO [main] ipc.RpcExecutor: RpcExecutor name using fifo as call queue; numCallQueues=3; maxQueueLength=300; handlerCount=30 2021-11-04 11:34:08,084 INFO [main] ipc.RpcExecutor: RpcExecutor name using fifo as call queue; numCallQueues=2; maxQueueLength=300; handlerCount=20 2021-11-04 11:34:08,084 INFO [main] ipc.RpcExecutor: RpcExecutor name using fifo as call queue; numCallQueues=1; maxQueueLength=300; handlerCount=3 2021-11-04 11:34:08,095 INFO [main] ipc.RpcServer: master/C3HM1/172.27.0.2:16000: started 10 reader(s) listening on port=16000 2021-11-04 11:34:08,238 INFO [main] metrics.MetricRegistries: Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2021-11-04 11:34:08,888 INFO [main] fs.HFileSystem: Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2021-11-04 11:34:08,891 INFO [main] fs.HFileSystem: Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2021-11-04 11:34:09,007 INFO [main] zookeeper.RecoverableZooKeeper: Process identifier=master:16000 connecting to ZooKeeper ensemble=C3hb-zk:11181 2021-11-04 11:34:09,014 INFO [main] zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT 2021-11-04 11:34:09,014 INFO [main] zookeeper.ZooKeeper: Client environment:host.name=C3HM1 2021-11-04 11:34:09,014 INFO [main] zookeeper.ZooKeeper: Client environment:java.version=1.8.0_271 2021-11-04 11:34:09,014 INFO [main] zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 2021-11-04 11:34:09,014 INFO [main] zookeeper.ZooKeeper: Client environment:java.home=/home/java/jdk1.8.0_271/jre 2021-11-04 11:34:09,014 INFO [main] zookeeper.ZooKeeper: 4j-1.2.17.jar:/home/evaluation/hbase-1.7.1/bin/../lib/metrics-core-2.2.0.jar:/home/evaluation/hbase-1.7.1/bin/../lib/metrics-core-3.1.2.jar:/home/evaluation/hbase-1.7.1/bin/../lib/netty-all-4.1.8.Final.jar:/home/evaluation/hbase-1.7.1/bin/../lib/paranamer-2.3.jar:/home/evaluation/hbase-1.7.1/bin/../lib/protobuf-java-2.5.0.jar:/home/evaluation/hbase-1.7.1/bin/../lib/servlet-api-2.5-6.1.14.jar:/home/evaluation/hbase-1.7.1/bin/../lib/servlet-api-2.5.jar:/home/evaluation/hbase-1.7.1/bin/../lib/slf4j-api-1.7.25.jar:/home/evaluation/hbase-1.7.1/bin/../lib/slf4j-log4j12-1.7.25.jar:/home/evaluation/hbase-1.7.1/bin/../lib/snappy-java-1.0.5.jar:/home/evaluation/hbase-1.7.1/bin/../lib/spymemcached-2.11.6.jar:/home/evaluation/hbase-1.7.1/bin/../lib/xmlenc-0.52.jar:/home/evaluation/hbase-1.7.1/bin/../lib/xz-1.0.jar:/home/evaluation/hbase-1.7.1/bin/../lib/zookeeper-3.4.10.jar 2021-11-04 11:34:09,014 INFO [main] zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2021-11-04 11:34:09,014 INFO [main] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 2021-11-04 11:34:09,014 INFO [main] zookeeper.ZooKeeper: Client environment:java.compiler= 2021-11-04 11:34:09,014 INFO [main] zookeeper.ZooKeeper: Client environment:os.name=Linux 2021-11-04 11:34:09,014 INFO [main] zookeeper.ZooKeeper: Client environment:os.arch=amd64 2021-11-04 11:34:09,014 INFO [main] zookeeper.ZooKeeper: Client environment:os.version=3.10.0-514.el7.x86_64 2021-11-04 11:34:09,014 INFO [main] zookeeper.ZooKeeper: Client environment:user.name=root 2021-11-04 11:34:09,014 INFO [main] zookeeper.ZooKeeper: Client environment:user.home=/root 2021-11-04 11:34:09,014 INFO [main] zookeeper.ZooKeeper: Client environment:user.dir=/home/evaluation/hbase-1.7.1 2021-11-04 11:34:09,015 INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=C3hb-zk:11181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@73511076 2021-11-04 11:34:09,030 INFO [main-SendThread(C3hb-zk:11181)] zookeeper.ClientCnxn: Opening socket connection to server C3hb-zk/172.27.0.8:11181. Will not attempt to authenticate using SASL (unknown error) 2021-11-04 11:34:09,033 INFO [main-SendThread(C3hb-zk:11181)] zookeeper.ClientCnxn: Socket connection established to C3hb-zk/172.27.0.8:11181, initiating session 2021-11-04 11:34:09,041 INFO [main-SendThread(C3hb-zk:11181)] zookeeper.ClientCnxn: Session establishment complete on server C3hb-zk/172.27.0.8:11181, sessionid = 0x10014d1be7a6435, negotiated timeout = 40000 2021-11-04 11:34:09,092 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2021-11-04 11:34:09,092 INFO [RpcServer.listener,port=16000] ipc.RpcServer: RpcServer.listener,port=16000: starting 2021-11-04 11:34:09,139 INFO [main] mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2021-11-04 11:34:09,142 INFO [main] http.HttpRequestLog: Http request log for http.requests.master is not defined 2021-11-04 11:34:09,149 INFO [main] http.HttpServer: Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2021-11-04 11:34:09,149 INFO [main] http.HttpServer: Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2021-11-04 11:34:09,151 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context master 2021-11-04 11:34:09,151 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2021-11-04 11:34:09,151 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2021-11-04 11:34:09,157 INFO [main] http.HttpServer: ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2021-11-04 11:34:09,167 INFO [main] http.HttpServer: Jetty bound to port 16010 2021-11-04 11:34:09,167 INFO [main] mortbay.log: jetty-6.1.26 2021-11-04 11:34:09,318 INFO [main] mortbay.log: Started SelectChannelConnector@0.0.0.0:16010 2021-11-04 11:34:09,323 INFO [main] hbase.ChoreService: Chore [ScheduledChore: Name: CompactedHFilesCleaner Period: 120000 Unit: MILLISECONDS] is enabled. 2021-11-04 11:34:09,324 INFO [main] master.HMaster: hbase.rootdir=hdfs://C3hb-hdfs:9000/hbase, hbase.cluster.distributed=true 2021-11-04 11:34:09,342 INFO [main] master.HMaster: Adding backup master ZNode /hbase/backup-masters/c3hm1,16000,1635996847735 2021-11-04 11:34:09,416 INFO [C3HM1:16000.activeMasterManager] master.ActiveMasterManager: Deleting ZNode for /hbase/backup-masters/c3hm1,16000,1635996847735 from backup master directory 2021-11-04 11:34:09,420 INFO [C3HM1:16000.activeMasterManager] master.ActiveMasterManager: Registered Active Master=c3hm1,16000,1635996847735 2021-11-04 11:34:09,422 INFO [C3HM1:16000.activeMasterManager] util.FSTableDescriptors: Attempting to repair HBase 1.7.0 tables, if any. 2021-11-04 11:34:09,432 INFO [master/C3HM1/172.27.0.2:16000] zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x29ee14d6 connecting to ZooKeeper ensemble=C3hb-zk:11181 2021-11-04 11:34:09,432 INFO [master/C3HM1/172.27.0.2:16000] zookeeper.ZooKeeper: Initiating client connection, connectString=C3hb-zk:11181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@9cc84e1 2021-11-04 11:34:09,433 INFO [master/C3HM1/172.27.0.2:16000-SendThread(C3hb-zk:11181)] zookeeper.ClientCnxn: Opening socket connection to server C3hb-zk/172.27.0.8:11181. Will not attempt to authenticate using SASL (unknown error) 2021-11-04 11:34:09,434 INFO [master/C3HM1/172.27.0.2:16000-SendThread(C3hb-zk:11181)] zookeeper.ClientCnxn: Socket connection established to C3hb-zk/172.27.0.8:11181, initiating session 2021-11-04 11:34:09,435 INFO [master/C3HM1/172.27.0.2:16000-SendThread(C3hb-zk:11181)] zookeeper.ClientCnxn: Session establishment complete on server C3hb-zk/172.27.0.8:11181, sessionid = 0x10014d1be7a6436, negotiated timeout = 40000 2021-11-04 11:34:09,436 INFO [master/C3HM1/172.27.0.2:16000] client.ZKConnectionRegistry: ClusterId read in ZooKeeper is null 2021-11-04 11:34:10,363 INFO [C3HM1:16000.activeMasterManager] util.FSUtils: Created version file at hdfs://C3hb-hdfs:9000/hbase with version=8 2021-11-04 11:34:10,875 INFO [C3HM1:16000.activeMasterManager] master.MasterFileSystem: BOOTSTRAP: creating hbase:meta region 2021-11-04 11:34:10,902 INFO [C3HM1:16000.activeMasterManager] regionserver.HRegion: creating HRegion hbase:meta HTD == 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}, {NAME => 'info', BLOOMFILTER => 'NONE', VERSIONS => '3', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', CACHE_DATA_IN_L1 => 'true', MIN_VERSIONS => '0', BLOCKCACHE => 'false', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} RootDir = hdfs://C3hb-hdfs:9000/hbase Table name == hbase:meta 2021-11-04 11:34:11,411 INFO [StoreOpener-1588230740-1] Configuration.deprecation: hbase.offheapcache.minblocksize is deprecated. Instead, use hbase.blockcache.minblocksize 2021-11-04 11:34:11,412 INFO [StoreOpener-1588230740-1] hfile.CacheConfig: Allocating LruBlockCache size=10.67 GB, blockSize=64 KB 2021-11-04 11:34:11,417 INFO [StoreOpener-1588230740-1] hfile.CacheConfig: Created cacheConfig for info: blockCache=LruBlockCache{blockCount=0, currentSize=8389120, freeSize=11444157952, maxSize=11452547072, heapSize=8389120, minSize=10879919104, minFactor=0.95, multiSize=5439959552, multiFactor=0.5, singleSize=2719979776, singleFactor=0.25}, cacheDataOnRead=false, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 2021-11-04 11:34:11,425 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration: size [minCompactSize:134217728, maxCompactSize:9223372036854775807, offPeakMaxCompactSize:9223372036854775807); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory 2021-11-04 11:34:11,434 INFO [StoreOpener-1588230740-1] regionserver.HStore: Store=info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, encoding=NONE, compression=NONE 2021-11-04 11:34:11,488 INFO [C3HM1:16000.activeMasterManager] regionserver.HRegion: Onlined 1588230740; next sequenceid=2 2021-11-04 11:34:11,491 INFO [StoreCloserThread-hbase:meta,,1.1588230740-1] regionserver.HStore: Closed info 2021-11-04 11:34:11,491 INFO [C3HM1:16000.activeMasterManager] regionserver.HRegion: Closed hbase:meta,,1.1588230740 2021-11-04 11:34:11,552 INFO [C3HM1:16000.activeMasterManager] fs.HFileSystem: Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2021-11-04 11:34:11,560 INFO [C3HM1:16000.activeMasterManager] coordination.SplitLogManagerCoordination: Found 0 orphan tasks and 0 rescan nodes 2021-11-04 11:34:11,560 INFO [C3HM1:16000.activeMasterManager] hbase.ChoreService: Chore [ScheduledChore: Name: SplitLogManager Timeout Monitor Period: 1000 Unit: MILLISECONDS] is enabled. 2021-11-04 11:34:11,582 INFO [C3HM1:16000.activeMasterManager] zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x2c2fe5b4 connecting to ZooKeeper ensemble=C3hb-zk:11181 2021-11-04 11:34:11,582 INFO [C3HM1:16000.activeMasterManager] zookeeper.ZooKeeper: Initiating client connection, connectString=C3hb-zk:11181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@4bcf3c27 2021-11-04 11:34:11,583 INFO [C3HM1:16000.activeMasterManager-SendThread(C3hb-zk:11181)] zookeeper.ClientCnxn: Opening socket connection to server C3hb-zk/172.27.0.8:11181. Will not attempt to authenticate using SASL (unknown error) 2021-11-04 11:34:11,584 INFO [C3HM1:16000.activeMasterManager-SendThread(C3hb-zk:11181)] zookeeper.ClientCnxn: Socket connection established to C3hb-zk/172.27.0.8:11181, initiating session 2021-11-04 11:34:11,586 INFO [C3HM1:16000.activeMasterManager-SendThread(C3hb-zk:11181)] zookeeper.ClientCnxn: Session establishment complete on server C3hb-zk/172.27.0.8:11181, sessionid = 0x10014d1be7a6439, negotiated timeout = 40000 2021-11-04 11:34:11,619 INFO [C3HM1:16000.activeMasterManager] balancer.StochasticLoadBalancer: loading config 2021-11-04 11:34:11,623 INFO [C3HM1:16000.activeMasterManager] balancer.StochasticLoadBalancer: Loaded config; maxSteps=1000000, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, etc., maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemstoreSizeCostFunction, StoreFileCostFunction] etc. 2021-11-04 11:34:11,666 INFO [C3HM1:16000.activeMasterManager] master.HMaster: Server active/primary master=c3hm1,16000,1635996847735, sessionid=0x10014d1be7a6435, setting cluster-up flag (Was=false) 2021-11-04 11:34:11,672 INFO [master/C3HM1/172.27.0.2:16000] regionserver.HRegionServer: ClusterId : c59d43c9-9411-4358-a930-ab0e819bf6cb 2021-11-04 11:34:11,694 INFO [C3HM1:16000.activeMasterManager] procedure.ZKProcedureUtil: Clearing all procedure znodes: /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2021-11-04 11:34:11,706 INFO [C3HM1:16000.activeMasterManager] procedure.ZKProcedureUtil: Clearing all procedure znodes: /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2021-11-04 11:34:11,711 WARN [C3HM1:16000.activeMasterManager] snapshot.SnapshotManager: Couldn't delete working snapshot directory: hdfs://C3hb-hdfs:9000/hbase/.hbase-snapshot/.tmp 2021-11-04 11:34:11,746 INFO [C3HM1:16000.activeMasterManager] master.MasterCoprocessorHost: System coprocessor loading is enabled 2021-11-04 11:34:11,765 INFO [C3HM1:16000.activeMasterManager] procedure2.ProcedureExecutor: Starting procedure executor threads=65 2021-11-04 11:34:11,766 INFO [C3HM1:16000.activeMasterManager] wal.WALProcedureStore: Starting WAL Procedure Store lease recovery 2021-11-04 11:34:11,775 WARN [C3HM1:16000.activeMasterManager] util.CommonFSUtils: Your Hadoop installation does not include the StreamCapabilities class from HDFS-11644, so we will skip checking if any FSDataOutputStreams actually support hflush/hsync. If you are running on top of HDFS this probably just means you have an older version and this can be ignored. If you are running on top of an alternate FileSystem implementation you should manually verify that hflush and hsync are implemented; otherwise you risk data loss and hard to diagnose errors when our assumptions are violated. 2021-11-04 11:34:11,779 INFO [C3HM1:16000.activeMasterManager] wal.WALProcedureStore: Lease acquired for flushLogId: 1 2021-11-04 11:34:11,789 INFO [C3HM1:16000.activeMasterManager] cleaner.DirScanPool: Cleaner pool size is 16 2021-11-04 11:34:11,790 INFO [C3HM1:16000.activeMasterManager] cleaner.CleanerChore: initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2021-11-04 11:34:11,820 INFO [C3HM1:16000.activeMasterManager] cleaner.CleanerChore: initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2021-11-04 11:34:11,820 INFO [C3HM1:16000.activeMasterManager] cleaner.LogCleaner: Creating OldWALs cleaners with size=2 2021-11-04 11:34:11,821 INFO [C3HM1:16000.activeMasterManager] hbase.ChoreService: Chore [ScheduledChore: Name: LogsCleaner Period: 600000 Unit: MILLISECONDS] is enabled. 2021-11-04 11:34:11,824 INFO [C3HM1:16000.activeMasterManager] cleaner.CleanerChore: initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2021-11-04 11:34:11,827 INFO [C3HM1:16000.activeMasterManager] cleaner.CleanerChore: initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2021-11-04 11:34:11,828 INFO [C3HM1:16000.activeMasterManager] cleaner.CleanerChore: initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2021-11-04 11:34:11,831 INFO [C3HM1:16000.activeMasterManager] hbase.ChoreService: Chore [ScheduledChore: Name: HFileCleaner Period: 600000 Unit: MILLISECONDS] is enabled. 2021-11-04 11:34:11,831 INFO [C3HM1:16000.activeMasterManager] master.HMaster: Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it." 2021-11-04 11:34:11,831 INFO [C3HM1:16000.activeMasterManager] hbase.ChoreService: Chore [ScheduledChore: Name: SnapshotCleaner Period: 1800000 Unit: MILLISECONDS] is enabled. 2021-11-04 11:34:11,838 INFO [C3HM1:16000.activeMasterManager] hbase.ChoreService: Chore [ScheduledChore: Name: ReplicationZKNodeCleanerChore Period: 600000 Unit: MILLISECONDS] is enabled. 2021-11-04 11:34:11,839 INFO [C3HM1:16000.activeMasterManager] master.ServerManager: Waiting on RegionServer count=0 to settle; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2021-11-04 11:34:12,624 INFO [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000] master.ServerManager: Registering server=c3rs1,16020,1635996850597 2021-11-04 11:34:12,641 INFO [C3HM1:16000.activeMasterManager] master.ServerManager: Waiting on RegionServer count=1 to settle; waited=802ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2021-11-04 11:34:13,812 INFO [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000] master.ServerManager: Registering server=c3rs2,16020,1635996852060 2021-11-04 11:34:13,844 INFO [C3HM1:16000.activeMasterManager] master.ServerManager: Waiting on RegionServer count=2 to settle; waited=2005ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2021-11-04 11:34:15,250 INFO [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000] master.ServerManager: Registering server=c3rs3,16020,1635996853475 2021-11-04 11:34:15,297 INFO [C3HM1:16000.activeMasterManager] master.ServerManager: Waiting on RegionServer count=3 to settle; waited=3458ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2021-11-04 11:34:16,801 INFO [C3HM1:16000.activeMasterManager] master.ServerManager: Finished wait on RegionServer count=3; waited=4962ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2021-11-04 11:34:16,810 INFO [C3HM1:16000.activeMasterManager] master.MasterFileSystem: Log folder hdfs://C3hb-hdfs:9000/hbase/WALs/c3rs1,16020,1635996850597 belongs to an existing region server 2021-11-04 11:34:17,629 INFO [C3HM1:16000.activeMasterManager] zookeeper.MetaTableLocator: Deleting hbase:meta region location in ZooKeeper 2021-11-04 11:34:17,653 INFO [C3HM1:16000.activeMasterManager] master.AssignmentManager: Setting node as OFFLINED in ZooKeeper for region {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2021-11-04 11:34:17,668 INFO [C3HM1:16000.activeMasterManager] zookeeper.ZKTableStateManager: Moving table hbase:meta state from null to ENABLED 2021-11-04 11:34:17,677 INFO [C3HM1:16000.activeMasterManager] master.AssignmentManager: Assigning hbase:meta,,1.1588230740 to c3rs2,16020,1635996852060 2021-11-04 11:34:17,678 INFO [C3HM1:16000.activeMasterManager] master.RegionStates: Transition {1588230740 state=OFFLINE, ts=1635996857653, server=null} to {1588230740 state=PENDING_OPEN, ts=1635996857677, server=c3rs2,16020,1635996852060} 2021-11-04 11:34:17,805 INFO [C3HM1:16000.activeMasterManager] master.ServerManager: AssignmentManager hasn't finished failover cleanup; waiting 2021-11-04 11:34:17,826 INFO [AM.ZK.Worker-pool8-t1] master.RegionStates: Transition {1588230740 state=PENDING_OPEN, ts=1635996857677, server=c3rs2,16020,1635996852060} to {1588230740 state=OPENING, ts=1635996857826, server=c3rs2,16020,1635996852060} 2021-11-04 11:34:59,976 INFO [zk-event-processor-pool1-t1] zookeeper.RegionServerTracker: RegionServer ephemeral node deleted, processing expiration [c3rs2,16020,1635996852060] 2021-11-04 11:38:11,588 INFO [Idle-Rpc-Conn-Sweeper-pool4-t1] ipc.AbstractRpcClient: Cleanup idle connection to C3RS2:16020 2021-11-04 11:39:11,418 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=8.00 MB, freeSize=10.66 GB, max=10.67 GB, blockCount=0, accesses=0, hits=0, hitRatio=0, cachingAccesses=0, cachingHits=0, cachingHitsRatio=0,evictions=29, evicted=0, evictedPerRun=0.0 2021-11-04 11:44:11,417 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=8.00 MB, freeSize=10.66 GB, max=10.67 GB, blockCount=0, accesses=0, hits=0, hitRatio=0, cachingAccesses=0, cachingHits=0, cachingHitsRatio=0,evictions=59, evicted=0, evictedPerRun=0.0