Thu Mar 24 01:43:10 CST 2022 Starting master on C1HM2 core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 512460 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1048576 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) unlimited virtual memory (kbytes, -v) unlimited file locks (-x) unlimited 2022-03-24 01:43:19,045 INFO [main] master.HMaster: STARTING service HMaster 2022-03-24 01:43:19,068 INFO [main] util.VersionInfo: HBase 2.4.8 2022-03-24 01:43:19,068 INFO [main] util.VersionInfo: Source code repository git://apurtell-ltm.internal.salesforce.com/Users/apurtell/tmp/RM/hbase revision=f844d09157d9dce6c54fcd53975b7a45865ee9ac 2022-03-24 01:43:19,069 INFO [main] util.VersionInfo: Compiled by apurtell on Wed Oct 27 08:48:57 PDT 2021 2022-03-24 01:43:19,069 INFO [main] util.VersionInfo: From source with checksum f575ee5f53f2f967078b59695c66493807de773ea5bd8dbf33b5574d8141d6d9720858a993b91466f188276b00c2a1cfc1a4b8fb7e1b9e65f2d99d62fa7de054 2022-03-24 01:43:21,525 INFO [main] util.ServerCommandLine: hbase.tmp.dir: /home/hbuser/evaluation/hbase-2.4.8/tmp 2022-03-24 01:43:21,526 INFO [main] util.ServerCommandLine: hbase.rootdir: hdfs://C1hb-hdfs:9000/hbase 2022-03-24 01:43:21,526 INFO [main] util.ServerCommandLine: hbase.cluster.distributed: true 2022-03-24 01:43:21,527 INFO [main] util.ServerCommandLine: hbase.zookeeper.quorum: C1hb-zk 2022-03-24 01:43:21,530 INFO [main] util.ServerCommandLine: env:PATH=/home/hbuser/java/jdk1.8.0_271/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2022-03-24 01:43:21,531 INFO [main] util.ServerCommandLine: env:HBASE_LOGFILE=hbase-root-master-C1HM2.log 2022-03-24 01:43:21,531 INFO [main] util.ServerCommandLine: env:FAV_OPTS=-Xbootclasspath/a:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar -javaagent:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar=useFav=true,useMsgid=false,jdkMsg=false,jdkFile=true,recordPhase=false,recordPath=/home/hbuser/hb244-fav-rst/,dataPaths=/home/hbuser/evaluation/hbase-2.4.8/tmp:/home/hbuser/evaluation/hbase-2.4.8/conf,cacheDir=/home/hbuser/HB244-CacheFolder,favHome=/home/hbuser,hdfsApi=true,zkApi=true,forHbase=true,currentCrash=/home/hbuser/hb244curCrash,controllerSocket=124.16.138.61:14900,strictCheck=false,mapSize=10000,wordSize=64,covPath=/home/hbuser/fuzzcov,covIncludes=org/apache/hadoop/hbase,aflAllow=/home/hbuser/evaluation/hbase-2.4.8/allowlist,aflDeny=/home/hbuser/evaluation/hbase-2.4.8/denylist,aflPort=12181 2022-03-24 01:43:21,532 INFO [main] util.ServerCommandLine: env:MSG_OPT=false 2022-03-24 01:43:21,532 INFO [main] util.ServerCommandLine: env:HBASE_ZOOKEEPER_OPTS= -Xbootclasspath/a:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar -javaagent:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar=useFav=false,hbaseRpc=true -Dfile.encoding=UTF8 -Duser.timezone=GMT+08 2022-03-24 01:43:21,533 INFO [main] util.ServerCommandLine: env:FAV_HOME=/home/hbuser 2022-03-24 01:43:21,534 INFO [main] util.ServerCommandLine: env:HBASE_REGIONSERVER_OPTS= -Xbootclasspath/a:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar -javaagent:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar=useFav=true,useMsgid=false,jdkMsg=false,jdkFile=true,recordPhase=false,recordPath=/home/hbuser/hb244-fav-rst/,dataPaths=/home/hbuser/evaluation/hbase-2.4.8/tmp:/home/hbuser/evaluation/hbase-2.4.8/conf,cacheDir=/home/hbuser/HB244-CacheFolder,favHome=/home/hbuser,hdfsApi=true,zkApi=true,forHbase=true,currentCrash=/home/hbuser/hb244curCrash,controllerSocket=124.16.138.61:14900,strictCheck=false,mapSize=10000,wordSize=64,covPath=/home/hbuser/fuzzcov,covIncludes=org/apache/hadoop/hbase,aflAllow=/home/hbuser/evaluation/hbase-2.4.8/allowlist,aflDeny=/home/hbuser/evaluation/hbase-2.4.8/denylist,aflPort=12181 -Dfile.encoding=UTF8 -Duser.timezone=GMT+08 2022-03-24 01:43:21,535 INFO [main] util.ServerCommandLine: env:JAVA_HOME=/home/hbuser/fav-jre-inst 2022-03-24 01:43:21,535 INFO [main] util.ServerCommandLine: env:TERM=xterm 2022-03-24 01:43:21,536 INFO [main] util.ServerCommandLine: env:JRE_HOME=/home/hbuser/java/jdk1.8.0_271/jre 2022-03-24 01:43:21,536 INFO [main] util.ServerCommandLine: env:HBASE_SHELL_OPTS= -Xbootclasspath/a:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar -javaagent:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar=useFav=false,hbaseRpc=true -Dfile.encoding=UTF8 -Duser.timezone=GMT+08 2022-03-24 01:43:21,537 INFO [main] util.ServerCommandLine: env:JVM_PID=181 2022-03-24 01:43:21,537 INFO [main] util.ServerCommandLine: env:TIME_OPTS=-Dfile.encoding=UTF8 -Duser.timezone=GMT+08 2022-03-24 01:43:21,538 INFO [main] util.ServerCommandLine: env:HBASE_REST_OPTS= -Xbootclasspath/a:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar -javaagent:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar=useFav=false,hbaseRpc=true -Dfile.encoding=UTF8 -Duser.timezone=GMT+08 2022-03-24 01:43:21,539 INFO [main] util.ServerCommandLine: env:PWD=/home/hbuser/evaluation/hbase-2.4.8 2022-03-24 01:43:21,539 INFO [main] util.ServerCommandLine: env:JRUBY_OPTS= -Xbootclasspath/a:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar -javaagent:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar=useFav=false,hbaseRpc=true 2022-03-24 01:43:21,540 INFO [main] util.ServerCommandLine: env:HBASE_ROOT_LOGGER=INFO,RFA 2022-03-24 01:43:21,540 INFO [main] util.ServerCommandLine: env:PHOS_OPTS=-Xbootclasspath/a:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar -javaagent:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar=useFav=false,hbaseRpc=true 2022-03-24 01:43:21,541 INFO [main] util.ServerCommandLine: env:HBASE_ENV_INIT=true 2022-03-24 01:43:21,541 INFO [main] util.ServerCommandLine: env:HBASE_IDENT_STRING=root 2022-03-24 01:43:21,542 INFO [main] util.ServerCommandLine: env:HBASE_ZNODE_FILE=/tmp/hbase-root-master.znode 2022-03-24 01:43:21,542 INFO [main] util.ServerCommandLine: env:HBASE_LOG_PREFIX=hbase-root-master-C1HM2 2022-03-24 01:43:21,543 INFO [main] util.ServerCommandLine: env:HBASE_LOG_DIR=/home/hbuser/evaluation/hbase-2.4.8/bin/../logs 2022-03-24 01:43:21,543 INFO [main] util.ServerCommandLine: env:OLDPWD=/ 2022-03-24 01:43:21,544 INFO [main] util.ServerCommandLine: env:USER=root 2022-03-24 01:43:21,544 INFO [main] util.ServerCommandLine: env:HBASE_MASTER_OPTS= -Xbootclasspath/a:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar -javaagent:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar=useFav=true,useMsgid=false,jdkMsg=false,jdkFile=true,recordPhase=false,recordPath=/home/hbuser/hb244-fav-rst/,dataPaths=/home/hbuser/evaluation/hbase-2.4.8/tmp:/home/hbuser/evaluation/hbase-2.4.8/conf,cacheDir=/home/hbuser/HB244-CacheFolder,favHome=/home/hbuser,hdfsApi=true,zkApi=true,forHbase=true,currentCrash=/home/hbuser/hb244curCrash,controllerSocket=124.16.138.61:14900,strictCheck=false,mapSize=10000,wordSize=64,covPath=/home/hbuser/fuzzcov,covIncludes=org/apache/hadoop/hbase,aflAllow=/home/hbuser/evaluation/hbase-2.4.8/allowlist,aflDeny=/home/hbuser/evaluation/hbase-2.4.8/denylist,aflPort=12181 -Dfile.encoding=UTF8 -Duser.timezone=GMT+08 2022-03-24 01:43:21,546 INFO [main] util.ServerCommandLine: py-java-1.0.5.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/spymemcached-2.12.2.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/stax2-api-3.1.4.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/woodstox-core-5.0.3.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/xmlenc-0.52.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/zookeeper-3.5.7.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/zookeeper-jute-3.5.7.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/client-facing-thirdparty/htrace-core4-4.2.0-incubating.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/client-facing-thirdparty/log4j-1.2.17.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.30.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/client-facing-thirdparty/slf4j-log4j12-1.7.30.jar 2022-03-24 01:43:21,547 INFO [main] util.ServerCommandLine: env:HBASE_MANAGES_ZK=false 2022-03-24 01:43:21,548 INFO [main] util.ServerCommandLine: env:HBASE_AUTOSTART_FILE=/tmp/hbase-root-master.autostart 2022-03-24 01:43:21,548 INFO [main] util.ServerCommandLine: env:SED=sed 2022-03-24 01:43:21,549 INFO [main] util.ServerCommandLine: env:HOSTNAME=C1HM2 2022-03-24 01:43:21,549 INFO [main] util.ServerCommandLine: env:GREP=grep 2022-03-24 01:43:21,550 INFO [main] util.ServerCommandLine: env:HBASE_NICENESS=0 2022-03-24 01:43:21,550 INFO [main] util.ServerCommandLine: env:HBASE_SECURITY_LOGGER=INFO,RFAS 2022-03-24 01:43:21,551 INFO [main] util.ServerCommandLine: Xbootclasspath/a:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar -javaagent:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar=useFav=true,useMsgid=false,jdkMsg=false,jdkFile=true,recordPhase=false,recordPath=/home/hbuser/hb244-fav-rst/,dataPaths=/home/hbuser/evaluation/hbase-2.4.8/tmp:/home/hbuser/evaluation/hbase-2.4.8/conf,cacheDir=/home/hbuser/HB244-CacheFolder,favHome=/home/hbuser,hdfsApi=true,zkApi=true,forHbase=true,currentCrash=/home/hbuser/hb244curCrash,controllerSocket=124.16.138.61:14900,strictCheck=false,mapSize=10000,wordSize=64,covPath=/home/hbuser/fuzzcov,covIncludes=org/apache/hadoop/hbase,aflAllow=/home/hbuser/evaluation/hbase-2.4.8/allowlist,aflDeny=/home/hbuser/evaluation/hbase-2.4.8/denylist,aflPort=12181 -Dfile.encoding=UTF8 -Duser.timezone=GMT+08 -Dhbase.log.dir=/home/hbuser/evaluation/hbase-2.4.8/bin/../logs -Dhbase.log.file=hbase-root-master-C1HM2.log -Dhbase.home.dir=/home/hbuser/evaluation/hbase-2.4.8/bin/.. -Dhbase.id.str=root -Dhbase.root.logger=INFO,RFA -Dhbase.security.logger=INFO,RFAS 2022-03-24 01:43:21,552 INFO [main] util.ServerCommandLine: env:HBASE_HOME=/home/hbuser/evaluation/hbase-2.4.8/bin/.. 2022-03-24 01:43:21,552 INFO [main] util.ServerCommandLine: env:HBASE_THRIFT_OPTS= -Xbootclasspath/a:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar -javaagent:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar=useFav=false,hbaseRpc=true -Dfile.encoding=UTF8 -Duser.timezone=GMT+08 2022-03-24 01:43:21,553 INFO [main] util.ServerCommandLine: eb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: 2022-03-24 01:43:21,554 INFO [main] util.ServerCommandLine: env:SHLVL=1 2022-03-24 01:43:21,554 INFO [main] util.ServerCommandLine: env:HOME=/root 2022-03-24 01:43:21,555 INFO [main] util.ServerCommandLine: env:MALLOC_ARENA_MAX=4 2022-03-24 01:43:21,555 INFO [main] util.ServerCommandLine: vmName=Java HotSpot(TM) 64-Bit Server VM, vmVendor=Oracle Corporation, vmVersion=25.271-b09 2022-03-24 01:43:21,557 INFO [main] util.ServerCommandLine: path/a:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar, -javaagent:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar=useFav=true,useMsgid=false,jdkMsg=false,jdkFile=true,recordPhase=false,recordPath=/home/hbuser/hb244-fav-rst/,dataPaths=/home/hbuser/evaluation/hbase-2.4.8/tmp:/home/hbuser/evaluation/hbase-2.4.8/conf,cacheDir=/home/hbuser/HB244-CacheFolder,favHome=/home/hbuser,hdfsApi=true,zkApi=true,forHbase=true,currentCrash=/home/hbuser/hb244curCrash,controllerSocket=124.16.138.61:14900,strictCheck=false,mapSize=10000,wordSize=64,covPath=/home/hbuser/fuzzcov,covIncludes=org/apache/hadoop/hbase,aflAllow=/home/hbuser/evaluation/hbase-2.4.8/allowlist,aflDeny=/home/hbuser/evaluation/hbase-2.4.8/denylist,aflPort=12181, -Dfile.encoding=UTF8, -Duser.timezone=GMT+08, -Dhbase.log.dir=/home/hbuser/evaluation/hbase-2.4.8/bin/../logs, -Dhbase.log.file=hbase-root-master-C1HM2.log, -Dhbase.home.dir=/home/hbuser/evaluation/hbase-2.4.8/bin/.., -Dhbase.id.str=root, -Dhbase.root.logger=INFO,RFA, -Dhbase.security.logger=INFO,RFAS] 2022-03-24 01:43:27,686 INFO [main] metrics.MetricRegistries: Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2022-03-24 01:43:28,175 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2022-03-24 01:43:38,037 INFO [main] regionserver.RSRpcServices: master/C1HM2:16000 server-side Connection retries=45 2022-03-24 01:43:38,633 INFO [main] ipc.RpcExecutor: Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=3, maxQueueLength=300, handlerCount=30 2022-03-24 01:43:38,817 INFO [main] ipc.RpcExecutor: Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=300, handlerCount=20 2022-03-24 01:43:38,820 INFO [main] ipc.RWQueueRpcExecutor: priority.RWQ.Fifo writeQueues=1 writeHandlers=2 readQueues=1 readHandlers=18 scanQueues=0 scanHandlers=0 2022-03-24 01:43:38,821 INFO [main] ipc.RpcExecutor: Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=300, handlerCount=3 2022-03-24 01:43:38,821 INFO [main] ipc.RpcExecutor: Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=300, handlerCount=1 2022-03-24 01:43:40,936 INFO [main] ipc.RpcServerFactory: Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2022-03-24 01:43:41,704 INFO [main] Configuration.deprecation: hbase.ipc.server.reservoir.initial.buffer.size is deprecated. Instead, use hbase.server.allocator.buffer.size 2022-03-24 01:43:41,705 INFO [main] Configuration.deprecation: hbase.ipc.server.reservoir.initial.max is deprecated. Instead, use hbase.server.allocator.max.buffer.count 2022-03-24 01:43:43,671 INFO [main] ipc.NettyRpcServer: Bind to /172.25.0.3:16000 2022-03-24 01:43:54,611 INFO [main] fs.HFileSystem: Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2022-03-24 01:43:54,621 INFO [main] fs.HFileSystem: Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2022-03-24 01:43:54,916 INFO [main] zookeeper.RecoverableZooKeeper: Process identifier=master:16000 connecting to ZooKeeper ensemble=C1hb-zk:11181 2022-03-24 01:43:55,038 INFO [main] zookeeper.ZooKeeper: Client environment:zookeeper.version=3.5.7-f0fdd52973d373ffd9c86b81d99842dc2c7f660e, built on 02/10/2020 11:30 GMT 2022-03-24 01:43:55,039 INFO [main] zookeeper.ZooKeeper: Client environment:host.name=C1HM2 2022-03-24 01:43:55,039 INFO [main] zookeeper.ZooKeeper: Client environment:java.version=1.8.0_271 2022-03-24 01:43:55,040 INFO [main] zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 2022-03-24 01:43:55,040 INFO [main] zookeeper.ZooKeeper: Client environment:java.home=/home/hbuser/fav-jre-inst/jre 2022-03-24 01:43:55,041 INFO [main] zookeeper.ZooKeeper: /hbase-2.4.8/bin/../lib/spymemcached-2.12.2.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/stax2-api-3.1.4.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/woodstox-core-5.0.3.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/xmlenc-0.52.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/zookeeper-3.5.7.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/zookeeper-jute-3.5.7.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/client-facing-thirdparty/htrace-core4-4.2.0-incubating.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/client-facing-thirdparty/log4j-1.2.17.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.30.jar:/home/hbuser/evaluation/hbase-2.4.8/bin/../lib/client-facing-thirdparty/slf4j-log4j12-1.7.30.jar:/home/hbuser/Phosphor-0.0.5-SNAPSHOT.jar 2022-03-24 01:43:55,042 INFO [main] zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2022-03-24 01:43:55,042 INFO [main] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 2022-03-24 01:43:55,042 INFO [main] zookeeper.ZooKeeper: Client environment:java.compiler= 2022-03-24 01:43:55,043 INFO [main] zookeeper.ZooKeeper: Client environment:os.name=Linux 2022-03-24 01:43:55,043 INFO [main] zookeeper.ZooKeeper: Client environment:os.arch=amd64 2022-03-24 01:43:55,044 INFO [main] zookeeper.ZooKeeper: Client environment:os.version=3.10.0-514.el7.x86_64 2022-03-24 01:43:55,045 INFO [main] zookeeper.ZooKeeper: Client environment:user.name=root 2022-03-24 01:43:55,045 INFO [main] zookeeper.ZooKeeper: Client environment:user.home=/root 2022-03-24 01:43:55,046 INFO [main] zookeeper.ZooKeeper: Client environment:user.dir=/home/hbuser/evaluation/hbase-2.4.8 2022-03-24 01:43:55,046 INFO [main] zookeeper.ZooKeeper: Client environment:os.memory.free=1491MB 2022-03-24 01:43:55,047 INFO [main] zookeeper.ZooKeeper: Client environment:os.memory.max=30688MB 2022-03-24 01:43:55,047 INFO [main] zookeeper.ZooKeeper: Client environment:os.memory.total=2004MB 2022-03-24 01:43:55,085 INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=C1hb-zk:11181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@4ac1e8d4 2022-03-24 01:43:55,167 INFO [main] common.X509Util: Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation 2022-03-24 01:43:55,232 INFO [main] zookeeper.ClientCnxnSocket: jute.maxbuffer value is 4194304 Bytes 2022-03-24 01:43:55,350 INFO [main] zookeeper.ClientCnxn: zookeeper.request.timeout value is 0. feature enabled= 2022-03-24 01:43:55,450 INFO [main-SendThread(C1hb-zk:11181)] zookeeper.ClientCnxn: Opening socket connection to server C1hb-zk/172.25.0.8:11181. Will not attempt to authenticate using SASL (unknown error) 2022-03-24 01:43:55,462 INFO [main-SendThread(C1hb-zk:11181)] zookeeper.ClientCnxn: Socket connection established, initiating session, client: /172.25.0.3:51784, server: C1hb-zk/172.25.0.8:11181 2022-03-24 01:43:55,527 INFO [main-SendThread(C1hb-zk:11181)] zookeeper.ClientCnxn: Session establishment complete on server C1hb-zk/172.25.0.8:11181, sessionid = 0x1013caeb566e679, negotiated timeout = 40000 2022-03-24 01:43:56,391 INFO [main] util.log: Logging initialized @45341ms to org.apache.hbase.thirdparty.org.eclipse.jetty.util.log.Slf4jLog 2022-03-24 01:43:58,304 INFO [main] http.HttpServer: Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2022-03-24 01:43:58,310 INFO [main] http.HttpServer: Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2022-03-24 01:43:58,318 INFO [main] http.HttpServer: Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2022-03-24 01:43:58,338 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context master 2022-03-24 01:43:58,339 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2022-03-24 01:43:58,339 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2022-03-24 01:43:58,412 INFO [main] http.HttpServer: ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2022-03-24 01:43:59,145 INFO [main] http.HttpServer: Jetty bound to port 16010 2022-03-24 01:43:59,165 INFO [main] server.Server: jetty-9.4.41.v20210516; built: 2021-05-16T23:56:28.993Z; git: 98607f93c7833e7dc59489b13f3cb0a114fb9f4c; jvm 1.8.0_271-b09 2022-03-24 01:43:59,528 INFO [main] http.SecurityHeadersFilter: Added security headers filter 2022-03-24 01:43:59,564 INFO [main] handler.ContextHandler: Started o.a.h.t.o.e.j.s.ServletContextHandler@1e21d524{logs,/logs,file:///home/hbuser/evaluation/hbase-2.4.8/logs/,AVAILABLE} 2022-03-24 01:43:59,565 INFO [main] http.SecurityHeadersFilter: Added security headers filter 2022-03-24 01:43:59,566 INFO [main] handler.ContextHandler: Started o.a.h.t.o.e.j.s.ServletContextHandler@748a5ba5{static,/static,file:///home/hbuser/evaluation/hbase-2.4.8/hbase-webapps/static/,AVAILABLE} 2022-03-24 01:44:00,287 INFO [main] webapp.StandardDescriptorProcessor: NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2022-03-24 01:44:00,435 INFO [main] server.session: DefaultSessionIdManager workerName=node0 2022-03-24 01:44:00,436 INFO [main] server.session: No SessionScavenger set, using defaults 2022-03-24 01:44:00,459 INFO [main] server.session: node0 Scavenging every 660000ms 2022-03-24 01:44:00,517 INFO [main] http.SecurityHeadersFilter: Added security headers filter 2022-03-24 01:44:01,157 INFO [main] handler.ContextHandler: Started o.a.h.t.o.e.j.w.WebAppContext@128b5064{master,/,file:///home/hbuser/evaluation/hbase-2.4.8/hbase-webapps/master/,AVAILABLE}{file:/home/hbuser/evaluation/hbase-2.4.8/hbase-webapps/master} 2022-03-24 01:44:01,335 INFO [main] server.AbstractConnector: Started ServerConnector@63e147dd{HTTP/1.1, (http/1.1)}{0.0.0.0:16010} 2022-03-24 01:44:01,336 INFO [main] server.Server: Started @50285ms 2022-03-24 01:44:01,389 INFO [main] master.HMaster: hbase.rootdir=hdfs://C1hb-hdfs:9000/hbase, hbase.cluster.distributed=true 2022-03-24 01:44:01,697 INFO [master/C1HM2:16000:becomeActiveMaster] master.HMaster: Adding backup master ZNode /hbase/backup-masters/c1hm2,16000,1648057402387 2022-03-24 01:44:02,423 INFO [master/C1HM2:16000:becomeActiveMaster] master.ActiveMasterManager: Another master is the active master, c1hm1,16000,1648057399830; waiting to become the next active master 2022-03-24 01:44:14,890 INFO [master/C1HM2:16000] regionserver.HRegionServer: ClusterId : 74a7a08a-3ffc-4349-8641-dfd2091e8d45 2022-03-24 01:47:17,999 INFO [master/C1HM2:16000:becomeActiveMaster] master.ActiveMasterManager: Deleting ZNode for /hbase/backup-masters/c1hm2,16000,1648057402387 from backup master directory 2022-03-24 01:47:18,032 INFO [master/C1HM2:16000:becomeActiveMaster] master.ActiveMasterManager: Registered as active master=c1hm2,16000,1648057402387 2022-03-24 01:47:18,079 INFO [master/C1HM2:16000:becomeActiveMaster] regionserver.ChunkCreator: Allocating data MemStoreChunkPool with chunk size 2 MB, max count 5523, initial count 0 2022-03-24 01:47:18,098 INFO [master/C1HM2:16000:becomeActiveMaster] regionserver.ChunkCreator: Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 6137, initial count 0 2022-03-24 01:47:21,664 INFO [master/C1HM2:16000:becomeActiveMaster] fs.HFileSystem: Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2022-03-24 01:47:23,641 INFO [ReadOnlyZKClient-C1hb-zk:11181@0x1bdd052e] zookeeper.ZooKeeper: Initiating client connection, connectString=C1hb-zk:11181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$120/421006975@0 2022-03-24 01:47:23,643 INFO [ReadOnlyZKClient-C1hb-zk:11181@0x1bdd052e] zookeeper.ClientCnxnSocket: jute.maxbuffer value is 4194304 Bytes 2022-03-24 01:47:23,644 INFO [ReadOnlyZKClient-C1hb-zk:11181@0x1bdd052e] zookeeper.ClientCnxn: zookeeper.request.timeout value is 0. feature enabled= 2022-03-24 01:47:23,647 INFO [ReadOnlyZKClient-C1hb-zk:11181@0x1bdd052e-SendThread(C1hb-zk:11181)] zookeeper.ClientCnxn: Opening socket connection to server C1hb-zk/172.25.0.8:11181. Will not attempt to authenticate using SASL (unknown error) 2022-03-24 01:47:23,649 INFO [ReadOnlyZKClient-C1hb-zk:11181@0x1bdd052e-SendThread(C1hb-zk:11181)] zookeeper.ClientCnxn: Socket connection established, initiating session, client: /172.25.0.3:52504, server: C1hb-zk/172.25.0.8:11181 2022-03-24 01:47:23,653 INFO [ReadOnlyZKClient-C1hb-zk:11181@0x1bdd052e-SendThread(C1hb-zk:11181)] zookeeper.ClientCnxn: Session establishment complete on server C1hb-zk/172.25.0.8:11181, sessionid = 0x1013caeb566e6b2, negotiated timeout = 40000 2022-03-24 01:47:24,113 INFO [master/C1HM2:16000:becomeActiveMaster] region.MasterRegion: Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2022-03-24 01:47:24,130 INFO [master/C1HM2:16000:becomeActiveMaster] region.MasterRegionFlusherAndCompactor: Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2022-03-24 01:47:24,508 INFO [master/C1HM2:16000:becomeActiveMaster] wal.WALFactory: Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2022-03-24 01:47:24,770 INFO [master/C1HM2:16000:becomeActiveMaster] region.MasterRegion: Renamed hdfs://C1hb-hdfs:9000/hbase/MasterData/WALs/c1hm1,16000,1648057399830 to hdfs://C1hb-hdfs:9000/hbase/MasterData/WALs/c1hm1,16000,1648057399830-dead as it is dead 2022-03-24 01:47:24,808 INFO [master/C1HM2:16000:becomeActiveMaster] util.RecoverLeaseFSUtils: Recover lease on dfs file hdfs://C1hb-hdfs:9000/hbase/MasterData/WALs/c1hm1,16000,1648057399830-dead/c1hm1%2C16000%2C1648057399830.1648057449042 2022-03-24 01:47:24,879 INFO [master/C1HM2:16000:becomeActiveMaster] util.RecoverLeaseFSUtils: Failed to recover lease, attempt=0 on file=hdfs://C1hb-hdfs:9000/hbase/MasterData/WALs/c1hm1,16000,1648057399830-dead/c1hm1%2C16000%2C1648057399830.1648057449042 after 70ms 2022-03-24 01:47:28,884 INFO [master/C1HM2:16000:becomeActiveMaster] util.RecoverLeaseFSUtils: Recovered lease, attempt=1 on file=hdfs://C1hb-hdfs:9000/hbase/MasterData/WALs/c1hm1,16000,1648057399830-dead/c1hm1%2C16000%2C1648057399830.1648057449042 after 4075ms 2022-03-24 01:47:28,891 INFO [master/C1HM2:16000:becomeActiveMaster] region.MasterRegion: Renamed hdfs://C1hb-hdfs:9000/hbase/MasterData/WALs/c1hm1,16000,1648057399830-dead/c1hm1%2C16000%2C1648057399830.1648057449042 to hdfs://C1hb-hdfs:9000/hbase/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/c1hm1%2C16000%2C1648057399830.1648057449042 2022-03-24 01:47:28,892 INFO [master/C1HM2:16000:becomeActiveMaster] region.MasterRegion: Delete empty local region wal dir hdfs://C1hb-hdfs:9000/hbase/MasterData/WALs/c1hm1,16000,1648057399830-dead 2022-03-24 01:47:29,253 INFO [master/C1HM2:16000:becomeActiveMaster] wal.AbstractFSWAL: WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=c1hm2%2C16000%2C1648057402387, suffix=, logDir=hdfs://C1hb-hdfs:9000/hbase/MasterData/WALs/c1hm2,16000,1648057402387, archiveDir=hdfs://C1hb-hdfs:9000/hbase/MasterData/oldWALs, maxLogs=10 2022-03-24 01:47:30,104 INFO [master/C1HM2:16000:becomeActiveMaster] wal.AbstractFSWAL: New WAL /hbase/MasterData/WALs/c1hm2,16000,1648057402387/c1hm2%2C16000%2C1648057402387.1648057649409 2022-03-24 01:47:31,412 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2022-03-24 01:47:32,067 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration: size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2022-03-24 01:47:32,219 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore: Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2022-03-24 01:47:32,251 INFO [master/C1HM2:16000:becomeActiveMaster] regionserver.HRegion: Replaying edits from hdfs://C1hb-hdfs:9000/hbase/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/c1hm1%2C16000%2C1648057399830.1648057449042 2022-03-24 01:47:33,140 INFO [master/C1HM2:16000:becomeActiveMaster] regionserver.HRegion: Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=124.13 KB heapSize=153.30 KB 2022-03-24 01:47:34,458 INFO [master/C1HM2:16000:becomeActiveMaster] regionserver.DefaultStoreFlusher: Flushed memstore data size=124.13 KB at sequenceid=294 (bloomFilter=true), to=hdfs://C1hb-hdfs:9000/hbase/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/39c5c9119661479fbba38a903e96c963 2022-03-24 01:47:34,882 INFO [master/C1HM2:16000:becomeActiveMaster] regionserver.StoreFileReader: Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 39c5c9119661479fbba38a903e96c963 2022-03-24 01:47:34,981 INFO [master/C1HM2:16000:becomeActiveMaster] regionserver.StoreFileReader: Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 39c5c9119661479fbba38a903e96c963 2022-03-24 01:47:34,982 INFO [master/C1HM2:16000:becomeActiveMaster] regionserver.HStore: Added hdfs://C1hb-hdfs:9000/hbase/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/39c5c9119661479fbba38a903e96c963, entries=338, sequenceid=294, filesize=129.8 K 2022-03-24 01:47:35,082 INFO [master/C1HM2:16000:becomeActiveMaster] regionserver.HRegion: Finished flush of dataSize ~124.13 KB/127107, heapSize ~153.27 KB/156952, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 1942ms, sequenceid=294, compaction requested=false; wal=null 2022-03-24 01:47:35,240 INFO [master/C1HM2:16000:becomeActiveMaster] regionserver.HRegion: Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=295; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10567511360, jitterRate=-0.015823811292648315}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2022-03-24 01:47:35,262 INFO [master/C1HM2:16000:becomeActiveMaster] region.MasterRegionFlusherAndCompactor: Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2022-03-24 01:47:35,640 INFO [master/C1HM2:16000:becomeActiveMaster] region.RegionProcedureStore: Starting the Region Procedure Store, number threads=16 2022-03-24 01:47:35,641 INFO [master/C1HM2:16000:becomeActiveMaster] procedure2.ProcedureExecutor: Starting 16 core workers (bigger of cpus/4 or 16) with max (burst) worker count=160 2022-03-24 01:47:35,661 INFO [master/C1HM2:16000:becomeActiveMaster] region.RegionProcedureStore: Starting Region Procedure Store lease recovery... 2022-03-24 01:47:35,672 INFO [master/C1HM2:16000:becomeActiveMaster] procedure2.ProcedureExecutor: Recovered RegionProcedureStore lease in 3 msec 2022-03-24 01:47:37,113 INFO [master/C1HM2:16000:becomeActiveMaster] procedure2.ProcedureExecutor: Loaded RegionProcedureStore in 1.4390 sec 2022-03-24 01:47:37,113 INFO [master/C1HM2:16000:becomeActiveMaster] procedure2.RemoteProcedureDispatcher: Instantiated, coreThreads=128 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2022-03-24 01:47:37,754 INFO [master/C1HM2:16000:becomeActiveMaster] master.RegionServerTracker: Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 2 possibly 'live' servers, and 0 'splitting'. 2022-03-24 01:47:37,868 INFO [master/C1HM2:16000:becomeActiveMaster] master.ServerManager: Registering regionserver=c1rs1,16020,1648057404041 2022-03-24 01:47:37,905 INFO [master/C1HM2:16000:becomeActiveMaster] master.ServerManager: Processing expiration of c1rs3,16020,1648057410351 on c1hm2,16000,1648057402387 2022-03-24 01:47:38,249 INFO [master/C1HM2:16000:becomeActiveMaster] assignment.AssignmentManager: Scheduled ServerCrashProcedure pid=33 for c1rs3,16020,1648057410351 (carryingMeta=true) c1rs3,16020,1648057410351/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@52da806[Write locks = 1, Read locks = 0], oldState=ONLINE. 2022-03-24 01:47:38,393 INFO [master/C1HM2:16000:becomeActiveMaster] balancer.BaseLoadBalancer: slop=0.001, systemTablesOnMaster=false 2022-03-24 01:47:38,437 INFO [master/C1HM2:16000:becomeActiveMaster] balancer.StochasticLoadBalancer: Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2022-03-24 01:47:38,522 INFO [master/C1HM2:16000:becomeActiveMaster] normalizer.RegionNormalizerWorker: Normalizer rate limit set to unlimited 2022-03-24 01:47:38,738 INFO [master/C1HM2:16000:becomeActiveMaster] master.HMaster: Active/primary master=c1hm2,16000,1648057402387, sessionid=0x1013caeb566e679, setting cluster-up flag (Was=true) 2022-03-24 01:47:39,056 WARN [master/C1HM2:16000:becomeActiveMaster] snapshot.SnapshotManager: Couldn't delete working snapshot directory: hdfs://C1hb-hdfs:9000/hbase/.hbase-snapshot/.tmp 2022-03-24 01:47:39,951 INFO [master/C1HM2:16000:becomeActiveMaster] procedure2.TimeoutExecutorThread: ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1648057689951 2022-03-24 01:47:39,966 INFO [master/C1HM2:16000:becomeActiveMaster] cleaner.DirScanPool: Cleaner pool size is 16 2022-03-24 01:47:39,980 INFO [master/C1HM2:16000:becomeActiveMaster] cleaner.CleanerChore: Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2022-03-24 01:47:39,984 INFO [master/C1HM2:16000:becomeActiveMaster] zookeeper.RecoverableZooKeeper: Process identifier=replicationLogCleaner connecting to ZooKeeper ensemble=C1hb-zk:11181 2022-03-24 01:47:39,985 INFO [master/C1HM2:16000:becomeActiveMaster] zookeeper.ZooKeeper: Initiating client connection, connectString=C1hb-zk:11181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@213ffe1b 2022-03-24 01:47:39,986 INFO [master/C1HM2:16000:becomeActiveMaster] zookeeper.ClientCnxnSocket: jute.maxbuffer value is 4194304 Bytes 2022-03-24 01:47:39,987 INFO [master/C1HM2:16000:becomeActiveMaster] zookeeper.ClientCnxn: zookeeper.request.timeout value is 0. feature enabled= 2022-03-24 01:47:39,988 INFO [master/C1HM2:16000:becomeActiveMaster] cleaner.CleanerChore: Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2022-03-24 01:47:39,989 INFO [master/C1HM2:16000:becomeActiveMaster-SendThread(C1hb-zk:11181)] zookeeper.ClientCnxn: Opening socket connection to server C1hb-zk/172.25.0.8:11181. Will not attempt to authenticate using SASL (unknown error) 2022-03-24 01:47:39,991 INFO [master/C1HM2:16000:becomeActiveMaster-SendThread(C1hb-zk:11181)] zookeeper.ClientCnxn: Socket connection established, initiating session, client: /172.25.0.3:52542, server: C1hb-zk/172.25.0.8:11181 2022-03-24 01:47:39,991 INFO [PEWorker-2] procedure.ServerCrashProcedure: Start pid=33, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure c1rs3,16020,1648057410351, splitWal=true, meta=true 2022-03-24 01:47:39,993 INFO [master/C1HM2:16000:becomeActiveMaster] cleaner.CleanerChore: Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2022-03-24 01:47:39,995 INFO [master/C1HM2:16000:becomeActiveMaster-SendThread(C1hb-zk:11181)] zookeeper.ClientCnxn: Session establishment complete on server C1hb-zk/172.25.0.8:11181, sessionid = 0x1013caeb566e6b3, negotiated timeout = 40000 2022-03-24 01:47:39,998 INFO [master/C1HM2:16000:becomeActiveMaster] cleaner.CleanerChore: Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2022-03-24 01:47:39,998 INFO [master/C1HM2:16000:becomeActiveMaster] cleaner.LogCleaner: Creating 2 old WALs cleaner threads 2022-03-24 01:47:40,003 INFO [master/C1HM2:16000:becomeActiveMaster] hbase.ChoreService: Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2022-03-24 01:47:40,031 INFO [master/C1HM2:16000:becomeActiveMaster] cleaner.CleanerChore: Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2022-03-24 01:47:40,032 INFO [master/C1HM2:16000:becomeActiveMaster] cleaner.CleanerChore: Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2022-03-24 01:47:40,042 INFO [PEWorker-2] procedure.ServerCrashProcedure: Splitting WALs pid=33, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure c1rs3,16020,1648057410351, splitWal=true, meta=true, isMeta: true 2022-03-24 01:47:40,059 INFO [PEWorker-2] master.SplitWALManager: c1rs3,16020,1648057410351 WAL count=1, meta=true 2022-03-24 01:47:40,062 INFO [master/C1HM2:16000:becomeActiveMaster] cleaner.CleanerChore: Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2022-03-24 01:47:40,067 INFO [master/C1HM2:16000:becomeActiveMaster] cleaner.CleanerChore: Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2022-03-24 01:47:40,078 INFO [master/C1HM2:16000:becomeActiveMaster] hbase.ChoreService: Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2022-03-24 01:47:40,079 INFO [master/C1HM2:16000:becomeActiveMaster] master.HMaster: Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2022-03-24 01:47:40,089 INFO [master/C1HM2:16000:becomeActiveMaster] hbase.ChoreService: Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2022-03-24 01:47:40,090 INFO [master/C1HM2:16000:becomeActiveMaster] hbase.ChoreService: Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2022-03-24 01:47:40,091 INFO [master/C1HM2:16000:becomeActiveMaster] master.HMaster: Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1648057442383, completionTime=-1 2022-03-24 01:47:40,091 INFO [master/C1HM2:16000:becomeActiveMaster] master.ServerManager: Waiting on regionserver count=1; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2022-03-24 01:47:40,099 INFO [PEWorker-2] procedure2.ProcedureExecutor: Initialized subprocedures=[{pid=34, ppid=33, state=RUNNABLE:ACQUIRE_SPLIT_WAL_WORKER; SplitWALProcedure c1rs3%2C16020%2C1648057410351.meta.1648057540980.meta}] 2022-03-24 01:47:40,230 INFO [PEWorker-2] procedure2.ProcedureExecutor: Initialized subprocedures=[{pid=35, ppid=34, state=RUNNABLE; SplitWALRemoteProcedure c1rs3%2C16020%2C1648057410351.meta.1648057540980.meta, worker=c1rs1,16020,1648057404041}] 2022-03-24 01:47:41,597 INFO [master/C1HM2:16000:becomeActiveMaster] master.ServerManager: Waiting on regionserver count=1; waited=1506ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1506ms 2022-03-24 01:47:43,103 INFO [master/C1HM2:16000:becomeActiveMaster] master.ServerManager: Waiting on regionserver count=1; waited=3012ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3012ms 2022-03-24 01:47:44,608 INFO [master/C1HM2:16000:becomeActiveMaster] master.ServerManager: Finished waiting on RegionServer count=1; waited=4517ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2022-03-24 01:47:44,627 WARN [master/C1HM2:16000:becomeActiveMaster] master.HMaster: hbase:meta,,1.1588230740 is NOT online; state={1588230740 state=OPEN, ts=1648057657694, server=c1rs3,16020,1648057410351}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2022-03-24 01:47:45,629 WARN [master/C1HM2:16000:becomeActiveMaster] master.HMaster: hbase:meta,,1.1588230740 is NOT online; state={1588230740 state=OPEN, ts=1648057657694, server=c1rs3,16020,1648057410351}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2022-03-24 01:47:46,010 INFO [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000] wal.WALSplitUtil: Moved hdfs://C1hb-hdfs:9000/hbase/WALs/c1rs3,16020,1648057410351-splitting/c1rs3%2C16020%2C1648057410351.meta.1648057540980.meta to hdfs://C1hb-hdfs:9000/hbase/oldWALs 2022-03-24 01:47:46,035 INFO [PEWorker-4] procedure2.ProcedureExecutor: Finished subprocedure pid=35, resume processing ppid=34 2022-03-24 01:47:46,037 INFO [PEWorker-4] procedure2.ProcedureExecutor: Finished pid=35, ppid=34, state=SUCCESS; SplitWALRemoteProcedure c1rs3%2C16020%2C1648057410351.meta.1648057540980.meta, worker=c1rs1,16020,1648057404041 in 5.7910 sec 2022-03-24 01:47:46,070 INFO [PEWorker-6] procedure2.ProcedureExecutor: Finished subprocedure pid=34, resume processing ppid=33 2022-03-24 01:47:46,071 INFO [PEWorker-6] procedure2.ProcedureExecutor: Finished pid=34, ppid=33, state=SUCCESS; SplitWALProcedure c1rs3%2C16020%2C1648057410351.meta.1648057540980.meta, worker=c1rs1,16020,1648057404041 in 5.9680 sec 2022-03-24 01:47:46,083 INFO [PEWorker-7] master.SplitLogManager: hdfs://C1hb-hdfs:9000/hbase/WALs/c1rs3,16020,1648057410351-splitting dir is empty, no logs to split. 2022-03-24 01:47:46,084 INFO [PEWorker-7] master.SplitWALManager: c1rs3,16020,1648057410351 WAL count=0, meta=true 2022-03-24 01:47:46,281 INFO [PEWorker-7] procedure2.ProcedureExecutor: Initialized subprocedures=[{pid=36, ppid=33, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2022-03-24 01:47:46,376 INFO [PEWorker-7] procedure.MasterProcedureScheduler: Took xlock for pid=36, ppid=33, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2022-03-24 01:47:46,381 INFO [PEWorker-7] assignment.TransitRegionStateProcedure: Starting pid=36, ppid=33, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2022-03-24 01:47:46,561 INFO [PEWorker-9] zookeeper.MetaTableLocator: Setting hbase:meta replicaId=0 location in ZooKeeper as c1rs1,16020,1648057404041, state=OPENING 2022-03-24 01:47:46,695 INFO [PEWorker-9] procedure2.ProcedureExecutor: Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 1588230740, server=c1rs1,16020,1648057404041}] 2022-03-24 01:47:47,630 WARN [master/C1HM2:16000:becomeActiveMaster] master.HMaster: hbase:meta,,1.1588230740 is NOT online; state={1588230740 state=OPENING, ts=1648057666695, server=c1rs1,16020,1648057404041}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2022-03-24 01:47:50,251 INFO [PEWorker-11] zookeeper.MetaTableLocator: Setting hbase:meta replicaId=0 location in ZooKeeper as c1rs1,16020,1648057404041, state=OPEN 2022-03-24 01:47:50,383 INFO [AsyncFSWAL-0-hdfs://C1hb-hdfs:9000/hbase/MasterData] wal.AbstractFSWAL: Slow sync cost: 120 ms, current pipeline: [DatanodeInfoWithStorage[172.25.0.7:9866,DS-c880b2eb-3656-4484-955e-8fa2177db65a,DISK]] 2022-03-24 01:47:50,391 INFO [PEWorker-11] procedure2.ProcedureExecutor: Finished subprocedure pid=37, resume processing ppid=36 2022-03-24 01:47:50,392 INFO [PEWorker-11] procedure2.ProcedureExecutor: Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 1588230740, server=c1rs1,16020,1648057404041 in 3.5660 sec 2022-03-24 01:47:50,413 INFO [PEWorker-13] procedure2.ProcedureExecutor: Finished subprocedure pid=36, resume processing ppid=33 2022-03-24 01:47:50,414 INFO [PEWorker-13] procedure2.ProcedureExecutor: Finished pid=36, ppid=33, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 4.1110 sec 2022-03-24 01:47:52,881 INFO [master/C1HM2:16000:becomeActiveMaster] assignment.AssignmentManager: Number of RegionServers=1 2022-03-24 01:47:52,882 INFO [master/C1HM2:16000:becomeActiveMaster] procedure2.TimeoutExecutorThread: ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1648057732882 2022-03-24 01:47:52,883 INFO [master/C1HM2:16000:becomeActiveMaster] procedure2.TimeoutExecutorThread: ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1648057792883 2022-03-24 01:47:52,884 INFO [master/C1HM2:16000:becomeActiveMaster] assignment.AssignmentManager: Joined the cluster in 1.2510 sec 2022-03-24 01:47:53,273 WARN [master/C1HM2:16000:becomeActiveMaster] master.TableStateManager: FAVMyInfo has no table state in hbase:meta, assuming ENABLED 2022-03-24 01:47:53,617 INFO [master/C1HM2:16000:becomeActiveMaster] hbase.MetaTableAccessor: Updated tableName=FAVMyInfo, state=ENABLED in hbase:meta 2022-03-24 01:47:53,979 INFO [master/C1HM2:16000:becomeActiveMaster] hbase.ChoreService: Chore ScheduledChore name=c1hm2,16000,1648057402387-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2022-03-24 01:47:53,980 INFO [master/C1HM2:16000:becomeActiveMaster] hbase.ChoreService: Chore ScheduledChore name=c1hm2,16000,1648057402387-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2022-03-24 01:47:53,980 INFO [PEWorker-14] procedure.ServerCrashProcedure: c1rs3,16020,1648057410351 had 2 regions 2022-03-24 01:47:53,981 INFO [master/C1HM2:16000:becomeActiveMaster] hbase.ChoreService: Chore ScheduledChore name=c1hm2,16000,1648057402387-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2022-03-24 01:47:53,989 INFO [PEWorker-14] procedure.ServerCrashProcedure: Splitting WALs pid=33, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure c1rs3,16020,1648057410351, splitWal=true, meta=true, isMeta: false 2022-03-24 01:47:54,005 INFO [PEWorker-14] master.SplitWALManager: c1rs3,16020,1648057410351 WAL count=1, meta=false 2022-03-24 01:47:54,007 INFO [PEWorker-14] procedure2.ProcedureExecutor: Initialized subprocedures=[{pid=38, ppid=33, state=RUNNABLE:ACQUIRE_SPLIT_WAL_WORKER; SplitWALProcedure c1rs3%2C16020%2C1648057410351.1648057468996}] 2022-03-24 01:47:54,016 INFO [master/C1HM2:16000:becomeActiveMaster] hbase.ChoreService: Chore ScheduledChore name=CatalogJanitor-C1HM2:16000, period=300000, unit=MILLISECONDS is enabled. 2022-03-24 01:47:54,019 INFO [master/C1HM2:16000:becomeActiveMaster] hbase.ChoreService: Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2022-03-24 01:47:54,020 INFO [PEWorker-14] procedure2.ProcedureExecutor: Initialized subprocedures=[{pid=39, ppid=38, state=RUNNABLE; SplitWALRemoteProcedure c1rs3%2C16020%2C1648057410351.1648057468996, worker=c1rs1,16020,1648057404041}] 2022-03-24 01:47:54,028 WARN [master/C1HM2:16000:becomeActiveMaster] master.HMaster: hbase:namespace,,1648057481362.d9896fe2aeffc6cbe2f1c821a55f3fae. is NOT online; state={d9896fe2aeffc6cbe2f1c821a55f3fae state=OPEN, ts=1648057672881, server=c1rs3,16020,1648057410351}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2022-03-24 01:47:54,119 WARN [master/C1HM2:16000.Chore.1] janitor.CatalogJanitor: unknown_server=c1rs3,16020,1648057410351/hbase:namespace,,1648057481362.d9896fe2aeffc6cbe2f1c821a55f3fae. 2022-03-24 01:47:55,030 WARN [master/C1HM2:16000:becomeActiveMaster] master.HMaster: hbase:namespace,,1648057481362.d9896fe2aeffc6cbe2f1c821a55f3fae. is NOT online; state={d9896fe2aeffc6cbe2f1c821a55f3fae state=OPEN, ts=1648057672881, server=c1rs3,16020,1648057410351}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2022-03-24 01:47:57,032 WARN [master/C1HM2:16000:becomeActiveMaster] master.HMaster: hbase:namespace,,1648057481362.d9896fe2aeffc6cbe2f1c821a55f3fae. is NOT online; state={d9896fe2aeffc6cbe2f1c821a55f3fae state=OPEN, ts=1648057672881, server=c1rs3,16020,1648057410351}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2022-03-24 01:47:58,378 INFO [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000] wal.WALSplitUtil: Moved hdfs://C1hb-hdfs:9000/hbase/WALs/c1rs3,16020,1648057410351-splitting/c1rs3%2C16020%2C1648057410351.1648057468996 to hdfs://C1hb-hdfs:9000/hbase/oldWALs 2022-03-24 01:47:58,400 INFO [PEWorker-3] procedure2.ProcedureExecutor: Finished subprocedure pid=39, resume processing ppid=38 2022-03-24 01:47:58,402 INFO [PEWorker-3] procedure2.ProcedureExecutor: Finished pid=39, ppid=38, state=SUCCESS; SplitWALRemoteProcedure c1rs3%2C16020%2C1648057410351.1648057468996, worker=c1rs1,16020,1648057404041 in 4.3650 sec 2022-03-24 01:47:58,423 INFO [PEWorker-1] procedure2.ProcedureExecutor: Finished subprocedure pid=38, resume processing ppid=33 2022-03-24 01:47:58,424 INFO [PEWorker-1] procedure2.ProcedureExecutor: Finished pid=38, ppid=33, state=SUCCESS; SplitWALProcedure c1rs3%2C16020%2C1648057410351.1648057468996, worker=c1rs1,16020,1648057404041 in 4.4040 sec 2022-03-24 01:47:58,435 INFO [PEWorker-2] master.SplitLogManager: hdfs://C1hb-hdfs:9000/hbase/WALs/c1rs3,16020,1648057410351-splitting dir is empty, no logs to split. 2022-03-24 01:47:58,436 INFO [PEWorker-2] master.SplitWALManager: c1rs3,16020,1648057410351 WAL count=0, meta=false 2022-03-24 01:47:58,454 INFO [PEWorker-2] procedure2.ProcedureExecutor: Initialized subprocedures=[{pid=40, ppid=33, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d9896fe2aeffc6cbe2f1c821a55f3fae, ASSIGN}] 2022-03-24 01:47:58,461 INFO [PEWorker-2] procedure.MasterProcedureScheduler: Took xlock for pid=40, ppid=33, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d9896fe2aeffc6cbe2f1c821a55f3fae, ASSIGN 2022-03-24 01:47:58,467 INFO [PEWorker-2] assignment.TransitRegionStateProcedure: Starting pid=40, ppid=33, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=d9896fe2aeffc6cbe2f1c821a55f3fae, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2022-03-24 01:47:58,707 INFO [PEWorker-4] assignment.RegionStateStore: pid=40 updating hbase:meta row=d9896fe2aeffc6cbe2f1c821a55f3fae, regionState=OPENING, regionLocation=c1rs1,16020,1648057404041 2022-03-24 01:47:58,725 INFO [PEWorker-4] procedure2.ProcedureExecutor: Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure d9896fe2aeffc6cbe2f1c821a55f3fae, server=c1rs1,16020,1648057404041}] 2022-03-24 01:47:59,268 INFO [PEWorker-8] assignment.RegionStateStore: pid=40 updating hbase:meta row=d9896fe2aeffc6cbe2f1c821a55f3fae, regionState=OPEN, openSeqNum=6, regionLocation=c1rs1,16020,1648057404041 2022-03-24 01:47:59,306 INFO [PEWorker-8] procedure2.ProcedureExecutor: Finished subprocedure pid=41, resume processing ppid=40 2022-03-24 01:47:59,308 INFO [PEWorker-8] procedure2.ProcedureExecutor: Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure d9896fe2aeffc6cbe2f1c821a55f3fae, server=c1rs1,16020,1648057404041 in 567 msec 2022-03-24 01:47:59,319 INFO [PEWorker-7] procedure2.ProcedureExecutor: Finished subprocedure pid=40, resume processing ppid=33 2022-03-24 01:47:59,319 INFO [PEWorker-10] procedure.ServerCrashProcedure: removed crashed server c1rs3,16020,1648057410351 after splitting done 2022-03-24 01:47:59,320 INFO [PEWorker-7] procedure2.ProcedureExecutor: Finished pid=40, ppid=33, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=d9896fe2aeffc6cbe2f1c821a55f3fae, ASSIGN in 852 msec 2022-03-24 01:47:59,362 INFO [PEWorker-10] procedure2.ProcedureExecutor: Finished pid=33, state=SUCCESS; ServerCrashProcedure c1rs3,16020,1648057410351, splitWal=true, meta=true in 21.4010 sec 2022-03-24 01:48:01,576 INFO [master/C1HM2:16000:becomeActiveMaster] master.HMaster: Master has completed initialization 43.542sec 2022-03-24 01:48:01,609 INFO [master/C1HM2:16000:becomeActiveMaster] quotas.MasterQuotaManager: Quota support disabled 2022-03-24 01:48:01,625 INFO [PEWorker-9] hbase.MetaTableAccessor: Updated tableName=FAVMyInfo, state=ENABLING in hbase:meta 2022-03-24 01:48:01,632 INFO [master/C1HM2:16000:becomeActiveMaster] slowlog.SlowLogMasterService: Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2022-03-24 01:48:01,632 INFO [master/C1HM2:16000:becomeActiveMaster] zookeeper.ZKWatcher: not a secure deployment, proceeding 2022-03-24 01:48:01,642 INFO [master/C1HM2:16000:becomeActiveMaster] hbase.ChoreService: Chore ScheduledChore name=c1hm2,16000,1648057402387-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2022-03-24 01:48:01,670 INFO [PEWorker-9] procedure2.ProcedureExecutor: Initialized subprocedures=[{pid=42, ppid=32, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=FAVMyInfo, region=64c556d99d19fc4b9eb5d61b39d60793, ASSIGN}] 2022-03-24 01:48:01,680 INFO [PEWorker-9] procedure.MasterProcedureScheduler: Took xlock for pid=42, ppid=32, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=FAVMyInfo, region=64c556d99d19fc4b9eb5d61b39d60793, ASSIGN 2022-03-24 01:48:01,685 INFO [PEWorker-9] assignment.TransitRegionStateProcedure: Starting pid=42, ppid=32, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=FAVMyInfo, region=64c556d99d19fc4b9eb5d61b39d60793, ASSIGN; state=CLOSED, location=c1rs1,16020,1648057404041; forceNewPlan=false, retain=false 2022-03-24 01:48:01,723 INFO [master/C1HM2:16000:becomeActiveMaster] hbase.ChoreService: Chore ScheduledChore name=c1hm2,16000,1648057402387-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2022-03-24 01:48:01,838 INFO [PEWorker-13] assignment.RegionStateStore: pid=42 updating hbase:meta row=64c556d99d19fc4b9eb5d61b39d60793, regionState=OPENING, regionLocation=c1rs1,16020,1648057404041 2022-03-24 01:48:01,856 INFO [PEWorker-13] procedure2.ProcedureExecutor: Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 64c556d99d19fc4b9eb5d61b39d60793, server=c1rs1,16020,1648057404041}] 2022-03-24 01:48:02,154 INFO [PEWorker-15] assignment.RegionStateStore: pid=42 updating hbase:meta row=64c556d99d19fc4b9eb5d61b39d60793, regionState=OPEN, openSeqNum=2, regionLocation=c1rs1,16020,1648057404041 2022-03-24 01:48:02,185 INFO [PEWorker-15] procedure2.ProcedureExecutor: Finished subprocedure pid=43, resume processing ppid=42 2022-03-24 01:48:02,186 INFO [PEWorker-15] procedure2.ProcedureExecutor: Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 64c556d99d19fc4b9eb5d61b39d60793, server=c1rs1,16020,1648057404041 in 318 msec 2022-03-24 01:48:02,196 INFO [PEWorker-14] procedure2.ProcedureExecutor: Finished subprocedure pid=42, resume processing ppid=32 2022-03-24 01:48:02,197 INFO [PEWorker-14] procedure2.ProcedureExecutor: Finished pid=42, ppid=32, state=SUCCESS; TransitRegionStateProcedure table=FAVMyInfo, region=64c556d99d19fc4b9eb5d61b39d60793, ASSIGN in 515 msec 2022-03-24 01:48:02,217 INFO [PEWorker-16] hbase.MetaTableAccessor: Updated tableName=FAVMyInfo, state=ENABLED in hbase:meta 2022-03-24 01:48:02,252 INFO [PEWorker-16] procedure2.ProcedureExecutor: Finished pid=32, state=SUCCESS; TruncateTableProcedure (table=FAVMyInfo preserveSplits=true) in 1 mins, 41.463 sec 2022-03-24 01:48:07,898 INFO [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000] master.HMaster: Client=hbuser//172.25.0.1 disable FAVMyInfo 2022-03-24 01:48:07,956 INFO [PEWorker-3] hbase.MetaTableAccessor: Updated tableName=FAVMyInfo, state=DISABLING in hbase:meta 2022-03-24 01:48:07,965 INFO [PEWorker-3] procedure.DisableTableProcedure: Set FAVMyInfo to state=DISABLING 2022-03-24 01:48:07,989 INFO [PEWorker-3] procedure2.ProcedureExecutor: Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=FAVMyInfo, region=64c556d99d19fc4b9eb5d61b39d60793, UNASSIGN}] 2022-03-24 01:48:07,994 INFO [PEWorker-3] procedure.MasterProcedureScheduler: Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=FAVMyInfo, region=64c556d99d19fc4b9eb5d61b39d60793, UNASSIGN 2022-03-24 01:48:07,997 INFO [PEWorker-3] assignment.RegionStateStore: pid=45 updating hbase:meta row=64c556d99d19fc4b9eb5d61b39d60793, regionState=CLOSING, regionLocation=c1rs1,16020,1648057404041 2022-03-24 01:48:08,029 INFO [PEWorker-3] procedure2.ProcedureExecutor: Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 64c556d99d19fc4b9eb5d61b39d60793, server=c1rs1,16020,1648057404041}] 2022-03-24 01:48:08,348 INFO [PEWorker-2] assignment.RegionStateStore: pid=45 updating hbase:meta row=64c556d99d19fc4b9eb5d61b39d60793, regionState=CLOSED 2022-03-24 01:48:08,379 INFO [PEWorker-2] procedure2.ProcedureExecutor: Finished subprocedure pid=46, resume processing ppid=45 2022-03-24 01:48:08,380 INFO [PEWorker-2] procedure2.ProcedureExecutor: Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 64c556d99d19fc4b9eb5d61b39d60793, server=c1rs1,16020,1648057404041 in 339 msec 2022-03-24 01:48:08,388 INFO [PEWorker-6] procedure2.ProcedureExecutor: Finished subprocedure pid=45, resume processing ppid=44 2022-03-24 01:48:08,389 INFO [PEWorker-6] procedure2.ProcedureExecutor: Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=FAVMyInfo, region=64c556d99d19fc4b9eb5d61b39d60793, UNASSIGN in 390 msec 2022-03-24 01:48:08,409 INFO [PEWorker-4] hbase.MetaTableAccessor: Updated tableName=FAVMyInfo, state=DISABLED in hbase:meta 2022-03-24 01:48:08,430 INFO [PEWorker-4] procedure.DisableTableProcedure: Set FAVMyInfo to state=DISABLED 2022-03-24 01:48:08,444 INFO [PEWorker-4] procedure2.ProcedureExecutor: Finished pid=44, state=SUCCESS; DisableTableProcedure table=FAVMyInfo in 521 msec 2022-03-24 01:48:11,161 INFO [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000] master.HMaster: Client=hbuser//172.25.0.1 delete FAVMyInfo 2022-03-24 01:48:11,777 WARN [PEWorker-8] procedure.DeleteTableProcedure: Deleting some vestigial 1 rows of FAVMyInfo from hbase:meta 2022-03-24 01:48:13,094 INFO [PEWorker-8] hbase.MetaTableAccessor: Deleted 1 regions from META 2022-03-24 01:48:13,110 INFO [PEWorker-8] hbase.MetaTableAccessor: Deleted table FAVMyInfo state from META 2022-03-24 01:48:13,135 INFO [PEWorker-8] procedure2.ProcedureExecutor: Finished pid=47, state=SUCCESS; DeleteTableProcedure table=FAVMyInfo in 1.9480 sec 2022-03-24 01:48:22,635 INFO [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000] master.HMaster: Client=hbuser//172.25.0.1 disable FAVMyInfo 2022-03-24 01:48:22,649 INFO [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000] master.HMaster: Client=hbuser//172.25.0.1 disable FAVMyInfo