2020-11-16 23:29:30,990 INFO [main] zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT 2020-11-16 23:29:30,991 INFO [main] zookeeper.ZooKeeper: Client environment:host.name=cvp21 2020-11-16 23:29:30,991 INFO [main] zookeeper.ZooKeeper: Client environment:java.version=1.8.0_265 2020-11-16 23:29:30,991 INFO [main] zookeeper.ZooKeeper: Client environment:java.vendor=AdoptOpenJDK 2020-11-16 23:29:30,991 INFO [main] zookeeper.ZooKeeper: Client environment:java.home=/cvpi/jdk/jre 2020-11-16 23:29:30,991 INFO [main] zookeeper.ZooKeeper: pi/hadoop/share/hadoop/yarn/hadoop-yarn-services-core-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-server-router-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-client-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-registry-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-api-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-services-api-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1.3.jar:/cvpi/hadoop/contrib/capacity-scheduler/*.jar::/cvpi/hadoop/etc/hadoop:/cvpi/hbase/lib/zkcli/jline-0.9.94.jar:/cvpi/jdk/lib/jmx_prometheus_javaagent.jar 2020-11-16 23:29:30,992 INFO [main] zookeeper.ZooKeeper: Client environment:java.library.path=/cvpi/hadoop/lib/native:/cvpi/hadoop/lib/native 2020-11-16 23:29:30,992 INFO [main] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 2020-11-16 23:29:30,992 INFO [main] zookeeper.ZooKeeper: Client environment:java.compiler= 2020-11-16 23:29:30,992 INFO [main] zookeeper.ZooKeeper: Client environment:os.name=Linux 2020-11-16 23:29:30,992 INFO [main] zookeeper.ZooKeeper: Client environment:os.arch=amd64 2020-11-16 23:29:30,992 INFO [main] zookeeper.ZooKeeper: Client environment:os.version=3.10.0-1062.12.1.el7.x86_64 2020-11-16 23:29:30,992 INFO [main] zookeeper.ZooKeeper: Client environment:user.name=cvp 2020-11-16 23:29:30,992 INFO [main] zookeeper.ZooKeeper: Client environment:user.home=/home/cvp 2020-11-16 23:29:30,992 INFO [main] zookeeper.ZooKeeper: Client environment:user.dir=/home/cvp 2020-11-16 23:29:30,994 INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=cvp21:2181,cvp22:2181,cvp23:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@3745e5c6 2020-11-16 23:29:31,043 INFO [main-SendThread(cvp21:2181)] zookeeper.ClientCnxn: Opening socket connection to server cvp21/192.168.2.21:2181. Will not attempt to authenticate using SASL (unknown error) 2020-11-16 23:29:31,048 INFO [main-SendThread(cvp21:2181)] zookeeper.ClientCnxn: Socket connection established to cvp21/192.168.2.21:2181, initiating session 2020-11-16 23:29:31,057 INFO [main-SendThread(cvp21:2181)] zookeeper.ClientCnxn: Session establishment complete on server cvp21/192.168.2.21:2181, sessionid = 0x1000be3d4bf6c33, negotiated timeout = 60000 Mon Nov 16 23:29:32 UTC 2020 Starting regionserver on cvp21 core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 208408 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 65536 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 208408 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited 2020-11-16 23:29:34,383 INFO [main] regionserver.HRegionServer: STARTING executorService HRegionServer 2020-11-16 23:29:34,384 INFO [main] util.VersionInfo: HBase 2.2.4 2020-11-16 23:29:34,384 INFO [main] util.VersionInfo: Source code repository file:///hbase-2.2.4 revision=Unknown 2020-11-16 23:29:34,384 INFO [main] util.VersionInfo: Compiled by root on Mon Jun 1 20:27:22 UTC 2020 2020-11-16 23:29:34,384 INFO [main] util.VersionInfo: From source with checksum 811a583f3df897a058639334f00f928d 2020-11-16 23:29:34,643 INFO [main] util.ServerCommandLine: hbase.tmp.dir: /data/hbase 2020-11-16 23:29:34,643 INFO [main] util.ServerCommandLine: hbase.rootdir: hdfs://mycluster/hbase 2020-11-16 23:29:34,643 INFO [main] util.ServerCommandLine: hbase.cluster.distributed: true 2020-11-16 23:29:34,644 INFO [main] util.ServerCommandLine: hbase.zookeeper.quorum: cvp21,cvp22,cvp23 2020-11-16 23:29:34,646 INFO [main] util.ServerCommandLine: env:PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/cvpi/bin:/cvpi/jdk/bin:/cvpi/zookeeper/bin:/cvpi/clickhouse/bin:/cvpi/hadoop/bin:/cvpi/hadoop/sbin:/cvpi/hbase/bin 2020-11-16 23:29:34,646 INFO [main] util.ServerCommandLine: env:HADOOP_CONF_DIR=/cvpi/hadoop/etc/hadoop 2020-11-16 23:29:34,646 INFO [main] util.ServerCommandLine: env:LOAD_BALANCER_IP= 2020-11-16 23:29:34,646 INFO [main] util.ServerCommandLine: env:HBASE_PID_DIR=/var/run/cvpi/hbase/pid 2020-11-16 23:29:34,646 INFO [main] util.ServerCommandLine: env:KUBECONFIG=/etc/kubernetes/kubeconfig.yaml 2020-11-16 23:29:34,646 INFO [main] util.ServerCommandLine: env:CVP_HOME=/cvpi 2020-11-16 23:29:34,646 INFO [main] util.ServerCommandLine: env:HBASE_REGIONSERVER_OPTS= -javaagent:/cvpi/jdk/lib/jmx_prometheus_javaagent.jar=7073:/cvpi/hbase/conf/hbase-jmx-exporter-config.yml 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:TERTIARY_HOSTNAME=cvp23 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:CVP_VERSION=2020.3.0 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:LD_LIBRARY_PATH=:/cvpi/hadoop/lib/native:/cvpi/hadoop/lib/native 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:LOGNAME=cvp 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:LOAD_BALANCER_PORT= 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:JVM_PID=19510 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:HBASE_REST_OPTS= 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:PWD=/home/cvp 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:CVP_DATA_DIR=/data 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:CVP_USER=cvp 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:CV_WIFI_HA_CLUSTER_IP= 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:HBASE_ROOT_LOGGER=INFO,RFA 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:SHELL=/bin/bash 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:HBASE_HEAPSIZE=9389 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:HBASE_ENV_INIT=true 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:CVP_INTERFACE=eth1 2020-11-16 23:29:34,647 INFO [main] util.ServerCommandLine: env:SECONDARY_HOST_IP=192.168.2.22 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:HBASE_MASTER_OPTS= -javaagent:/cvpi/jdk/lib/jmx_prometheus_javaagent.jar=7072:/cvpi/hbase/conf/hbase-jmx-exporter-config.yml 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:HBASE_MANAGES_ZK=false 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:HADOOP_HOME=/cvpi/hadoop 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:HBASE_NICENESS=0 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:HBASE_OPTS=-XX:+UseG1GC -XX:MaxGCPauseMillis=100 -XX:+ParallelRefProcEnabled -XX:+PerfDisableSharedMem -Djava.net.preferIPv4Stack=true -Dlog4j.appender.RFA.MaxBackupIndex=0 -Dlog4j.appender.RFAS.MaxBackupIndex=0 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/cvpi/hbase/logs/hbase-cvp-regionserver-cvp21.gc -javaagent:/cvpi/jdk/lib/jmx_prometheus_javaagent.jar=7073:/cvpi/hbase/conf/hbase-jmx-exporter-config.yml -Dhbase.log.dir=/cvpi/hbase/logs -Dhbase.log.file=hbase-cvp-regionserver-cvp21.log -Dhbase.home.dir=/cvpi/hbase -Dhbase.id.str=cvp -Dhbase.root.logger=INFO,RFA -Djava.library.path=/cvpi/hadoop/lib/native:/cvpi/hadoop/lib/native -Dhbase.security.logger=INFO,RFAS 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:HBASE_SECURITY_LOGGER=INFO,RFAS 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:PRIMARY_DEVICE_INTF_IP=10.81.45.213 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:SHLVL=3 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:HBASE_LOGFILE=hbase-cvp-regionserver-cvp21.log 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:CVP_LOG_LEVEL=INFO 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:TERTIARY_DEVICE_INTF_IP=10.81.45.219 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:JAVA_HOME=/cvpi/jdk 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:KUBE_CLUSTER_NETWORK=100.96.0.0/16 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:HBASE_STOP_TIMEOUT=250 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:CVP_MODE=multinode 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:LANG=en_US.UTF-8 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:KUBE_CLUSTER_DNS=100.96.128.10 2020-11-16 23:29:34,648 INFO [main] util.ServerCommandLine: env:SECONDARY_HOSTNAME=cvp22 2020-11-16 23:29:34,649 INFO [main] util.ServerCommandLine: env:TERTIARY_HOST_IP=192.168.2.23 2020-11-16 23:29:34,649 INFO [main] util.ServerCommandLine: env:PRIMARY_HOST_IP=192.168.2.21 2020-11-16 23:29:34,649 INFO [main] util.ServerCommandLine: env:HBASE_CLASSPATH=:/cvpi/hadoop/etc/hadoop 2020-11-16 23:29:34,649 INFO [main] util.ServerCommandLine: env:KUBE_SERVICE_NETWORK=100.96.128.0/17 2020-11-16 23:29:34,649 INFO [main] util.ServerCommandLine: env:CV_WIFI_ENABLED=no 2020-11-16 23:29:34,649 INFO [main] util.ServerCommandLine: env:SECONDARY_DEVICE_INTF_IP=10.81.45.216 2020-11-16 23:29:34,649 INFO [main] util.ServerCommandLine: env:PRIMARY_HOSTNAME=cvp21 2020-11-16 23:29:34,654 INFO [main] util.ServerCommandLine: env:HBASE_SERVER_JAAS_OPTS= 2020-11-16 23:29:34,654 INFO [main] util.ServerCommandLine: env:HBASE_IDENT_STRING=cvp 2020-11-16 23:29:34,654 INFO [main] util.ServerCommandLine: env:HBASE_ZNODE_FILE=/var/run/cvpi/hbase/pid/hbase-cvp-regionserver.znode 2020-11-16 23:29:34,654 INFO [main] util.ServerCommandLine: env:HBASE_LOG_PREFIX=hbase-cvp-regionserver-cvp21 2020-11-16 23:29:34,654 INFO [main] util.ServerCommandLine: env:HBASE_LOG_DIR=/cvpi/hbase/logs 2020-11-16 23:29:34,654 INFO [main] util.ServerCommandLine: env:USER=cvp 2020-11-16 23:29:34,655 INFO [main] util.ServerCommandLine: hed-2.12.2.jar:/cvpi/hbase/lib/stax2-api-3.1.4.jar:/cvpi/hbase/lib/validation-api-1.1.0.Final.jar:/cvpi/hbase/lib/woodstox-core-5.0.3.jar:/cvpi/hbase/lib/zookeeper-3.4.10.jar:/cvpi/hbase/lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/cvpi/hbase/lib/client-facing-thirdparty/commons-logging-1.2.jar:/cvpi/hbase/lib/client-facing-thirdparty/findbugs-annotations-1.3.9-1.jar:/cvpi/hbase/lib/client-facing-thirdparty/htrace-core4-4.2.0-incubating.jar:/cvpi/hbase/lib/client-facing-thirdparty/log4j-1.2.17.jar:/cvpi/hbase/lib/client-facing-thirdparty/slf4j-api-1.7.25.jar:/cvpi/hadoop/etc/hadoop:/cvpi/hadoop/share/hadoop/common/lib/*:/cvpi/hadoop/share/hadoop/common/*:/cvpi/hadoop/share/hadoop/hdfs:/cvpi/hadoop/share/hadoop/hdfs/lib/*:/cvpi/hadoop/share/hadoop/hdfs/*:/cvpi/hadoop/share/hadoop/mapreduce/lib/*:/cvpi/hadoop/share/hadoop/mapreduce/*:/cvpi/hadoop/share/hadoop/yarn/lib/*:/cvpi/hadoop/share/hadoop/yarn/*:/cvpi/hadoop/contrib/capacity-scheduler/*.jar::/cvpi/hadoop/etc/hadoop 2020-11-16 23:29:34,655 INFO [main] util.ServerCommandLine: env:SERVER_GC_OPTS=-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/cvpi/hbase/logs/hbase-cvp-regionserver-cvp21.gc 2020-11-16 23:29:34,655 INFO [main] util.ServerCommandLine: env:KUBE_POD_NETWORK=100.96.0.0/17 2020-11-16 23:29:34,655 INFO [main] util.ServerCommandLine: env:HBASE_LIBRARY_PATH=/cvpi/hadoop/lib/native 2020-11-16 23:29:34,655 INFO [main] util.ServerCommandLine: env:CVP_GROUP=cvp 2020-11-16 23:29:34,655 INFO [main] util.ServerCommandLine: env:HBASE_AUTOSTART_FILE=/var/run/cvpi/hbase/pid/hbase-cvp-regionserver.autostart 2020-11-16 23:29:34,655 INFO [main] util.ServerCommandLine: env:HBASE_THRIFT_OPTS= 2020-11-16 23:29:34,655 INFO [main] util.ServerCommandLine: env:HBASE_HOME=/cvpi/hbase 2020-11-16 23:29:34,655 INFO [main] util.ServerCommandLine: env:HOME=/home/cvp 2020-11-16 23:29:34,655 INFO [main] util.ServerCommandLine: env:MALLOC_ARENA_MAX=4 2020-11-16 23:29:34,655 INFO [main] util.ServerCommandLine: vmName=OpenJDK 64-Bit Server VM, vmVendor=AdoptOpenJDK, vmVersion=25.265-b01 2020-11-16 23:29:34,655 INFO [main] util.ServerCommandLine: vmInputArguments=[-Dproc_regionserver, -XX:OnOutOfMemoryError=kill -9 %p, -Xmx9389m, -XX:+UseG1GC, -XX:MaxGCPauseMillis=100, -XX:+ParallelRefProcEnabled, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -Dlog4j.appender.RFA.MaxBackupIndex=0, -Dlog4j.appender.RFAS.MaxBackupIndex=0, -verbose:gc, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -Xloggc:/cvpi/hbase/logs/hbase-cvp-regionserver-cvp21.gc, -javaagent:/cvpi/jdk/lib/jmx_prometheus_javaagent.jar=7073:/cvpi/hbase/conf/hbase-jmx-exporter-config.yml, -Dhbase.log.dir=/cvpi/hbase/logs, -Dhbase.log.file=hbase-cvp-regionserver-cvp21.log, -Dhbase.home.dir=/cvpi/hbase, -Dhbase.id.str=cvp, -Dhbase.root.logger=INFO,RFA, -Djava.library.path=/cvpi/hadoop/lib/native:/cvpi/hadoop/lib/native, -Dhbase.security.logger=INFO,RFAS] 2020-11-16 23:29:35,244 INFO [main] metrics.MetricRegistries: Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2020-11-16 23:29:35,376 WARN [main] util.FSUtils: Configuration "dfs.client.read.shortcircuit.skip.checksum" should not be set to true. HBase checksum doesn't require it, see https://issues.apache.org/jira/browse/HBASE-6868. 2020-11-16 23:29:35,526 INFO [main] regionserver.RSRpcServices: regionserver/cvp21:16201 server-side Connection retries=45 2020-11-16 23:29:35,542 INFO [main] ipc.RpcExecutor: Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=1024, handlerCount=16 2020-11-16 23:29:35,544 INFO [main] ipc.RpcExecutor: Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=1024, handlerCount=10 2020-11-16 23:29:35,545 INFO [main] ipc.RWQueueRpcExecutor: priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=9 scanQueues=0 scanHandlers=0 2020-11-16 23:29:35,545 INFO [main] ipc.RpcExecutor: Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=1024, handlerCount=3 2020-11-16 23:29:35,545 INFO [main] ipc.RpcExecutor: Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=1024, handlerCount=1 2020-11-16 23:29:35,744 INFO [main] ipc.RpcServerFactory: Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2020-11-16 23:29:35,813 INFO [main] io.ByteBufferPool: Created with bufferSize=64 KB and maxPoolSize=1 KB 2020-11-16 23:29:35,933 INFO [main] ipc.NettyRpcServer: Bind to /192.168.2.21:16201 2020-11-16 23:29:36,016 INFO [main] Configuration.deprecation: hbase.offheapcache.minblocksize is deprecated. Instead, use hbase.blockcache.minblocksize 2020-11-16 23:29:36,017 INFO [main] hfile.BlockCacheFactory: Allocating onheap LruBlockCache size=1.83 GB, blockSize=64 KB 2020-11-16 23:29:37,412 INFO [main] fs.HFileSystem: Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2020-11-16 23:29:37,419 INFO [main] fs.HFileSystem: Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2020-11-16 23:29:37,476 INFO [main] zookeeper.RecoverableZooKeeper: Process identifier=regionserver:16201 connecting to ZooKeeper ensemble=cvp21:2181,cvp22:2181,cvp23:2181 2020-11-16 23:29:37,484 INFO [main] zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT 2020-11-16 23:29:37,484 INFO [main] zookeeper.ZooKeeper: Client environment:host.name=cvp21 2020-11-16 23:29:37,484 INFO [main] zookeeper.ZooKeeper: Client environment:java.version=1.8.0_265 2020-11-16 23:29:37,484 INFO [main] zookeeper.ZooKeeper: Client environment:java.vendor=AdoptOpenJDK 2020-11-16 23:29:37,484 INFO [main] zookeeper.ZooKeeper: Client environment:java.home=/cvpi/jdk/jre 2020-11-16 23:29:37,484 INFO [main] zookeeper.ZooKeeper: hadoop-yarn-server-common-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-services-core-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-server-router-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-client-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-registry-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-api-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-services-api-3.1.3.jar:/cvpi/hadoop/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1.3.jar:/cvpi/hadoop/contrib/capacity-scheduler/*.jar::/cvpi/hadoop/etc/hadoop:/cvpi/jdk/lib/jmx_prometheus_javaagent.jar 2020-11-16 23:29:37,484 INFO [main] zookeeper.ZooKeeper: Client environment:java.library.path=/cvpi/hadoop/lib/native:/cvpi/hadoop/lib/native 2020-11-16 23:29:37,484 INFO [main] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 2020-11-16 23:29:37,484 INFO [main] zookeeper.ZooKeeper: Client environment:java.compiler= 2020-11-16 23:29:37,484 INFO [main] zookeeper.ZooKeeper: Client environment:os.name=Linux 2020-11-16 23:29:37,484 INFO [main] zookeeper.ZooKeeper: Client environment:os.arch=amd64 2020-11-16 23:29:37,484 INFO [main] zookeeper.ZooKeeper: Client environment:os.version=3.10.0-1062.12.1.el7.x86_64 2020-11-16 23:29:37,484 INFO [main] zookeeper.ZooKeeper: Client environment:user.name=cvp 2020-11-16 23:29:37,484 INFO [main] zookeeper.ZooKeeper: Client environment:user.home=/home/cvp 2020-11-16 23:29:37,484 INFO [main] zookeeper.ZooKeeper: Client environment:user.dir=/home/cvp 2020-11-16 23:29:37,485 INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=cvp21:2181,cvp22:2181,cvp23:2181 sessionTimeout=20000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@3766c667 2020-11-16 23:29:37,499 INFO [main-SendThread(cvp21:2181)] zookeeper.ClientCnxn: Opening socket connection to server cvp21/10.81.45.213:2181. Will not attempt to authenticate using SASL (unknown error) 2020-11-16 23:29:37,504 INFO [main-SendThread(cvp21:2181)] zookeeper.ClientCnxn: Socket connection established to cvp21/10.81.45.213:2181, initiating session 2020-11-16 23:29:37,513 INFO [main-SendThread(cvp21:2181)] zookeeper.ClientCnxn: Session establishment complete on server cvp21/10.81.45.213:2181, sessionid = 0x1000be3d4bf6c56, negotiated timeout = 60000 2020-11-16 23:29:37,579 INFO [main] util.log: Logging initialized @4815ms 2020-11-16 23:29:37,663 INFO [main] http.HttpRequestLog: Http request log for http.requests.regionserver is not defined 2020-11-16 23:29:37,679 INFO [main] http.HttpServer: Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2020-11-16 23:29:37,680 INFO [main] http.HttpServer: Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2020-11-16 23:29:37,680 INFO [main] http.HttpServer: Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2020-11-16 23:29:37,683 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2020-11-16 23:29:37,683 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2020-11-16 23:29:37,683 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2020-11-16 23:29:37,694 INFO [main] http.HttpServer: ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2020-11-16 23:29:37,716 INFO [main] http.HttpServer: Jetty bound to port 16301 2020-11-16 23:29:37,718 INFO [main] server.Server: jetty-9.3.27.v20190418, build timestamp: 2019-04-18T18:11:38Z, git hash: d3e249f86955d04bc646bb620905b7c1bc596a8d 2020-11-16 23:29:37,755 INFO [main] http.SecurityHeadersFilter: Added security headers filter 2020-11-16 23:29:37,759 INFO [main] handler.ContextHandler: Started o.e.j.s.ServletContextHandler@1dc2de84{/logs,file:///cvpi/hbase/logs/,AVAILABLE} 2020-11-16 23:29:37,759 INFO [main] http.SecurityHeadersFilter: Added security headers filter 2020-11-16 23:29:37,759 INFO [main] handler.ContextHandler: Started o.e.j.s.ServletContextHandler@58d63b16{/static,file:///cvpi/hbase/hbase-webapps/static/,AVAILABLE} 2020-11-16 23:29:37,979 INFO [main] webapp.StandardDescriptorProcessor: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet 2020-11-16 23:29:37,988 INFO [main] http.SecurityHeadersFilter: Added security headers filter 2020-11-16 23:29:38,005 INFO [main] handler.ContextHandler: Started o.e.j.w.WebAppContext@7f572c37{/,file:///cvpi/hbase/hbase-webapps/regionserver/,AVAILABLE}{file:/cvpi/hbase/hbase-webapps/regionserver} 2020-11-16 23:29:38,011 INFO [main] server.AbstractConnector: Started ServerConnector@44841b43{HTTP/1.1,[http/1.1]}{0.0.0.0:16301} 2020-11-16 23:29:38,011 INFO [main] server.Server: Started @5247ms 2020-11-16 23:29:38,038 INFO [regionserver/cvp21:16201] regionserver.HRegionServer: ClusterId : debc3c09-40af-432b-a167-a00c9f91e20c 2020-11-16 23:29:38,116 INFO [ReadOnlyZKClient-cvp21:2181,cvp22:2181,cvp23:2181@0x438b9f1b] zookeeper.ZooKeeper: Initiating client connection, connectString=cvp21:2181,cvp22:2181,cvp23:2181 sessionTimeout=20000 watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$42/1854950766@2b92ded8 2020-11-16 23:29:38,118 INFO [ReadOnlyZKClient-cvp21:2181,cvp22:2181,cvp23:2181@0x438b9f1b-SendThread(cvp21:2181)] zookeeper.ClientCnxn: Opening socket connection to server cvp21/10.81.45.213:2181. Will not attempt to authenticate using SASL (unknown error) 2020-11-16 23:29:38,119 INFO [ReadOnlyZKClient-cvp21:2181,cvp22:2181,cvp23:2181@0x438b9f1b-SendThread(cvp21:2181)] zookeeper.ClientCnxn: Socket connection established to cvp21/10.81.45.213:2181, initiating session 2020-11-16 23:29:38,122 INFO [ReadOnlyZKClient-cvp21:2181,cvp22:2181,cvp23:2181@0x438b9f1b-SendThread(cvp21:2181)] zookeeper.ClientCnxn: Session establishment complete on server cvp21/10.81.45.213:2181, sessionid = 0x1000be3d4bf6c57, negotiated timeout = 60000 2020-11-16 23:29:38,192 INFO [regionserver/cvp21:16201] regionserver.RegionServerCoprocessorHost: System coprocessor loading is enabled 2020-11-16 23:29:38,192 INFO [regionserver/cvp21:16201] regionserver.RegionServerCoprocessorHost: Table coprocessor loading is enabled 2020-11-16 23:29:38,202 INFO [regionserver/cvp21:16201] regionserver.HRegionServer: reportForDuty to master=cvp21,16000,1605473603163 with port=16201, startcode=1605569374704 2020-11-16 23:29:38,386 INFO [regionserver/cvp21:16201] wal.WALFactory: Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2020-11-16 23:29:38,610 INFO [regionserver/cvp21:16201] regionserver.MetricsRegionServerWrapperImpl: Computing regionserver metrics every 5000 milliseconds 2020-11-16 23:29:38,645 INFO [regionserver/cvp21:16201] regionserver.MemStoreFlusher: globalMemStoreLimit=3.7 G, globalMemStoreLimitLowMark=3.5 G, Offheap=false 2020-11-16 23:29:38,656 INFO [regionserver/cvp21:16201] throttle.PressureAwareCompactionThroughputController: Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2020-11-16 23:29:38,657 INFO [regionserver/cvp21:16201] hbase.ChoreService: Chore [ScheduledChore: Name: CompactionThroughputTuner Period: 60000 Unit: MILLISECONDS] is enabled. 2020-11-16 23:29:38,659 INFO [regionserver/cvp21:16201] regionserver.HRegionServer: CompactionChecker runs every PT10S 2020-11-16 23:29:38,703 INFO [regionserver/cvp21:16201] hbase.ChoreService: Chore [ScheduledChore: Name: CompactedHFilesCleaner Period: 120000 Unit: MILLISECONDS] is enabled. 2020-11-16 23:29:38,709 INFO [regionserver/cvp21:16201] hbase.ChoreService: Chore [ScheduledChore: Name: CompactionChecker Period: 10000 Unit: MILLISECONDS] is enabled. 2020-11-16 23:29:38,709 INFO [regionserver/cvp21:16201] hbase.ChoreService: Chore [ScheduledChore: Name: MemstoreFlusherChore Period: 10000 Unit: MILLISECONDS] is enabled. 2020-11-16 23:29:38,709 INFO [regionserver/cvp21:16201] hbase.ChoreService: Chore [ScheduledChore: Name: nonceCleaner Period: 360000 Unit: MILLISECONDS] is enabled. 2020-11-16 23:29:38,709 INFO [regionserver/cvp21:16201] hbase.ChoreService: Chore [ScheduledChore: Name: MovedRegionsCleaner for region cvp21,16201,1605569374704 Period: 120000 Unit: MILLISECONDS] is enabled. 2020-11-16 23:29:38,737 INFO [SplitLogWorker-cvp21:16201] regionserver.SplitLogWorker: SplitLogWorker cvp21,16201,1605569374704 starting 2020-11-16 23:29:38,742 INFO [regionserver/cvp21:16201] regionserver.HeapMemoryManager: Starting, tuneOn=false 2020-11-16 23:29:38,746 INFO [regionserver/cvp21:16201] hbase.ChoreService: Chore [ScheduledChore: Name: cvp21,16201,1605569374704-HeapMemoryTunerChore Period: 60000 Unit: MILLISECONDS] is enabled. 2020-11-16 23:29:38,751 INFO [regionserver/cvp21:16201] regionserver.ChunkCreator: Allocating data MemStoreChunkPool with chunk size 2 MB, max count 1690, initial count 0 2020-11-16 23:29:38,753 INFO [regionserver/cvp21:16201] regionserver.ChunkCreator: Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 1878, initial count 0 2020-11-16 23:29:38,759 INFO [SplitLogWorker-cvp21:16201] coordination.ZkSplitLogWorkerCoordination: worker cvp21,16201,1605569374704 acquired task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605566198473 2020-11-16 23:29:38,791 INFO [regionserver/cvp21:16201] regionserver.HRegionServer: Serving as cvp21,16201,1605569374704, RpcServer on cvp21/192.168.2.21:16201, sessionid=0x1000be3d4bf6c56 2020-11-16 23:29:38,796 INFO [regionserver/cvp21:16201] quotas.RegionServerRpcQuotaManager: Quota support disabled 2020-11-16 23:29:38,796 INFO [regionserver/cvp21:16201] quotas.RegionServerSpaceQuotaManager: Quota support disabled, not starting space quota manager. 2020-11-16 23:29:38,849 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.WALSplitter: Splitting WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566198473, size=128.0M (134221880 bytes) 2020-11-16 23:29:38,860 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] util.FSHDFSUtils: Recover lease on dfs file hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566198473 2020-11-16 23:29:38,879 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] util.FSHDFSUtils: Recovered lease, attempt=0 on file=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566198473 after 19ms 2020-11-16 23:29:38,980 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:39,085 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:39,105 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.WALSplitter: Open WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566198473 cost 257 ms 2020-11-16 23:29:39,284 INFO [SplitLogWorker-cvp21:16201] coordination.ZkSplitLogWorkerCoordination: worker cvp21,16201,1605569374704 acquired task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605567097516 2020-11-16 23:29:39,314 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.WALSplitter: Splitting WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567097516, size=129.1M (135397650 bytes) 2020-11-16 23:29:39,320 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] util.FSHDFSUtils: Recover lease on dfs file hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567097516 2020-11-16 23:29:39,323 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] util.FSHDFSUtils: Recovered lease, attempt=0 on file=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567097516 after 2ms 2020-11-16 23:29:39,335 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:39,348 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:39,353 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.WALSplitter: Open WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567097516 cost 39 ms 2020-11-16 23:29:39,378 INFO [Thread-23] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:39,479 INFO [Thread-25] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:39,512 INFO [Thread-30] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:39,515 INFO [Thread-26] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:39,515 INFO [Thread-34] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:39,516 INFO [Thread-33] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:41,387 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.OutputSink: 3 split writers finished; closing. 2020-11-16 23:29:41,414 INFO [Thread-39] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:41,414 INFO [Thread-40] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:41,548 INFO [split-log-closeStream-1] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/__test-table_cvp21__/22d0b8a2f061eb1ac72b4ca27d122e8d/recovered.edits/0000000000000002006-cvp21%2C16201%2C1605560366412.1605566198473.temp to hdfs://mycluster/hbase/data/default/__test-table_cvp21__/22d0b8a2f061eb1ac72b4ca27d122e8d/recovered.edits/0000000000000002007 2020-11-16 23:29:41,548 INFO [split-log-closeStream-3] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/__test-table_cvp22__/0bb85ddacc1f4a872bc5a7f3ebaa4878/recovered.edits/0000000000000002004-cvp21%2C16201%2C1605560366412.1605566198473.temp to hdfs://mycluster/hbase/data/default/__test-table_cvp22__/0bb85ddacc1f4a872bc5a7f3ebaa4878/recovered.edits/0000000000000002005 2020-11-16 23:29:41,548 INFO [split-log-closeStream-2] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/1a21a43d6ff9f33e59894a94c0613703/recovered.edits/0000000000260131004-cvp21%2C16201%2C1605560366412.1605566198473.temp to hdfs://mycluster/hbase/data/default/aeris_v2/1a21a43d6ff9f33e59894a94c0613703/recovered.edits/0000000000260134284 2020-11-16 23:29:41,557 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.WALSplitter: Processed 3285 edits across 3 regions cost 2451 ms; edits skipped=19012; WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566198473, size=128.0M, length=134221880, corrupted=false, progress failed=false 2020-11-16 23:29:41,563 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] coordination.ZkSplitLogWorkerCoordination: successfully transitioned task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605566198473 to final state DONE cvp21,16201,1605569374704 2020-11-16 23:29:41,563 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] handler.WALSplitterHandler: Worker cvp21,16201,1605569374704 done with task org.apache.hadoop.hbase.coordination.ZkSplitLogWorkerCoordination$ZkSplitTaskDetails@1dbeadc7 in 2798ms. Status = DONE 2020-11-16 23:29:41,605 INFO [SplitLogWorker-cvp21:16201] coordination.ZkSplitLogWorkerCoordination: worker cvp21,16201,1605569374704 acquired task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605567281316 2020-11-16 23:29:41,634 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.WALSplitter: Splitting WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567281316, size=128.8M (135037979 bytes) 2020-11-16 23:29:41,642 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] util.FSHDFSUtils: Recover lease on dfs file hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567281316 2020-11-16 23:29:41,644 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] util.FSHDFSUtils: Recovered lease, attempt=0 on file=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567281316 after 2ms 2020-11-16 23:29:41,662 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:41,667 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.WALSplitter: Open WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567281316 cost 33 ms 2020-11-16 23:29:41,760 INFO [Thread-44] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:41,764 INFO [Thread-45] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:41,818 INFO [Thread-43] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:41,818 INFO [Thread-47] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:41,889 INFO [Thread-46] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:41,889 INFO [Thread-49] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:41,926 INFO [Thread-48] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:43,845 INFO [regionserver/cvp21:16201] wal.AbstractFSWAL: WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=cvp21%2C16201%2C1605569374704, suffix=, logDir=hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704, archiveDir=hdfs://mycluster/hbase/oldWALs 2020-11-16 23:29:43,857 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.OutputSink: 3 split writers finished; closing. 2020-11-16 23:29:43,903 INFO [Thread-27] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:43,998 INFO [regionserver/cvp21:16201] wal.AbstractFSWAL: New WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569383863 2020-11-16 23:29:44,256 INFO [split-log-closeStream-3] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/recovered.edits/0000000000290096264-cvp21%2C16201%2C1605560366412.1605567097516.temp to hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/recovered.edits/0000000000290102943 2020-11-16 23:29:44,260 INFO [split-log-closeStream-1] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/__test-table_cvp21__/22d0b8a2f061eb1ac72b4ca27d122e8d/recovered.edits/0000000000000002014-cvp21%2C16201%2C1605560366412.1605567097516.temp to hdfs://mycluster/hbase/data/default/__test-table_cvp21__/22d0b8a2f061eb1ac72b4ca27d122e8d/recovered.edits/0000000000000002015 2020-11-16 23:29:44,260 INFO [split-log-closeStream-2] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/1a21a43d6ff9f33e59894a94c0613703/recovered.edits/0000000000260139769-cvp21%2C16201%2C1605560366412.1605567097516.temp to hdfs://mycluster/hbase/data/default/aeris_v2/1a21a43d6ff9f33e59894a94c0613703/recovered.edits/0000000000260146400 2020-11-16 23:29:44,263 INFO [Thread-28] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:44,305 INFO [split-log-closeStream-2] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/9816d2c4238016bd7836ffcfd49f49cc/recovered.edits/0000000000276711759-cvp21%2C16201%2C1605560366412.1605567097516.temp to hdfs://mycluster/hbase/data/default/aeris_v2/9816d2c4238016bd7836ffcfd49f49cc/recovered.edits/0000000000276718321 2020-11-16 23:29:44,325 INFO [split-log-closeStream-3] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/__test-table_cvp22__/0bb85ddacc1f4a872bc5a7f3ebaa4878/recovered.edits/0000000000000002012-cvp21%2C16201%2C1605560366412.1605567097516.temp to hdfs://mycluster/hbase/data/default/__test-table_cvp22__/0bb85ddacc1f4a872bc5a7f3ebaa4878/recovered.edits/0000000000000002013 2020-11-16 23:29:44,326 INFO [Thread-29] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:44,336 INFO [split-log-closeStream-1] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/recovered.edits/0000000000286177898-cvp21%2C16201%2C1605560366412.1605567097516.temp to hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/recovered.edits/0000000000286184602 2020-11-16 23:29:44,363 INFO [split-log-closeStream-3] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/154c3df7e6796980bbda638087f4a3dc/recovered.edits/0000000000261488312-cvp21%2C16201%2C1605560366412.1605567097516.temp to hdfs://mycluster/hbase/data/default/aeris_v2/154c3df7e6796980bbda638087f4a3dc/recovered.edits/0000000000261495059 2020-11-16 23:29:44,366 INFO [split-log-closeStream-2] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/__test-table_cvp23__/bac2756ecc315bfd457fb71d81dd3eb5/recovered.edits/0000000000000002026-cvp21%2C16201%2C1605560366412.1605567097516.temp to hdfs://mycluster/hbase/data/default/__test-table_cvp23__/bac2756ecc315bfd457fb71d81dd3eb5/recovered.edits/0000000000000002027 2020-11-16 23:29:44,368 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.WALSplitter: Processed 33334 edits across 8 regions cost 5012 ms; edits skipped=34752; WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567097516, size=129.1M, length=135397650, corrupted=false, progress failed=false 2020-11-16 23:29:44,371 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] coordination.ZkSplitLogWorkerCoordination: successfully transitioned task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605567097516 to final state DONE cvp21,16201,1605569374704 2020-11-16 23:29:44,371 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] handler.WALSplitterHandler: Worker cvp21,16201,1605569374704 done with task org.apache.hadoop.hbase.coordination.ZkSplitLogWorkerCoordination$ZkSplitTaskDetails@41160020 in 5085ms. Status = DONE 2020-11-16 23:29:44,384 INFO [Thread-60] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:44,392 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.OutputSink: 3 split writers finished; closing. 2020-11-16 23:29:44,447 INFO [SplitLogWorker-cvp21:16201] coordination.ZkSplitLogWorkerCoordination: worker cvp21,16201,1605569374704 acquired task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605566082047 2020-11-16 23:29:44,449 INFO [split-log-closeStream-3] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/recovered.edits/0000000000270768788-cvp21%2C16201%2C1605560366412.1605567281316.temp to hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/recovered.edits/0000000000270769986 2020-11-16 23:29:44,451 INFO [split-log-closeStream-1] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/recovered.edits/0000000000260420560-cvp21%2C16201%2C1605560366412.1605567281316.temp to hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/recovered.edits/0000000000260421777 2020-11-16 23:29:44,452 INFO [split-log-closeStream-2] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/1a21a43d6ff9f33e59894a94c0613703/recovered.edits/0000000000260147615-cvp21%2C16201%2C1605560366412.1605567281316.temp to hdfs://mycluster/hbase/data/default/aeris_v2/1a21a43d6ff9f33e59894a94c0613703/recovered.edits/0000000000260150345 2020-11-16 23:29:44,469 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.WALSplitter: Splitting WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566082047, size=32.5M (34081975 bytes) 2020-11-16 23:29:44,472 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] util.FSHDFSUtils: Recover lease on dfs file hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566082047 2020-11-16 23:29:44,474 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] util.FSHDFSUtils: Recovered lease, attempt=0 on file=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566082047 after 2ms 2020-11-16 23:29:44,489 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:44,492 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.WALSplitter: Open WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566082047 cost 23 ms 2020-11-16 23:29:44,502 INFO [split-log-closeStream-1] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/recovered.edits/0000000000286185849-cvp21%2C16201%2C1605560366412.1605567281316.temp to hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/recovered.edits/0000000000286187093 2020-11-16 23:29:44,505 INFO [split-log-closeStream-2] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/recovered.edits/0000000000259136877-cvp21%2C16201%2C1605560366412.1605567281316.temp to hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/recovered.edits/0000000000259136918 2020-11-16 23:29:44,508 INFO [split-log-closeStream-3] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/recovered.edits/0000000000290104118-cvp21%2C16201%2C1605560366412.1605567281316.temp to hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/recovered.edits/0000000000290105293 2020-11-16 23:29:44,539 INFO [split-log-closeStream-2] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/154c3df7e6796980bbda638087f4a3dc/recovered.edits/0000000000261496164-cvp21%2C16201%2C1605560366412.1605567281316.temp to hdfs://mycluster/hbase/data/default/aeris_v2/154c3df7e6796980bbda638087f4a3dc/recovered.edits/0000000000261497322 2020-11-16 23:29:44,543 INFO [split-log-closeStream-1] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/9816d2c4238016bd7836ffcfd49f49cc/recovered.edits/0000000000276719463-cvp21%2C16201%2C1605560366412.1605567281316.temp to hdfs://mycluster/hbase/data/default/aeris_v2/9816d2c4238016bd7836ffcfd49f49cc/recovered.edits/0000000000276720602 2020-11-16 23:29:44,543 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.WALSplitter: Processed 9909 edits across 8 regions cost 2876 ms; edits skipped=4050; WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567281316, size=128.8M, length=135037979, corrupted=false, progress failed=false 2020-11-16 23:29:44,549 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] coordination.ZkSplitLogWorkerCoordination: successfully transitioned task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605567281316 to final state DONE cvp21,16201,1605569374704 2020-11-16 23:29:44,549 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] handler.WALSplitterHandler: Worker cvp21,16201,1605569374704 done with task org.apache.hadoop.hbase.coordination.ZkSplitLogWorkerCoordination$ZkSplitTaskDetails@2aa5b285 in 2943ms. Status = DONE 2020-11-16 23:29:44,557 INFO [Thread-62] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:45,103 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.OutputSink: 3 split writers finished; closing. 2020-11-16 23:29:45,207 INFO [split-log-closeStream-1] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/1a21a43d6ff9f33e59894a94c0613703/recovered.edits/0000000000260126946-cvp21%2C16201%2C1605560366412.1605566082047.temp to hdfs://mycluster/hbase/data/default/aeris_v2/1a21a43d6ff9f33e59894a94c0613703/recovered.edits/0000000000260127721 2020-11-16 23:29:45,208 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.WALSplitter: Processed 776 edits across 1 regions cost 716 ms; edits skipped=4586; WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566082047, size=32.5M, length=34081975, corrupted=false, progress failed=false 2020-11-16 23:29:45,216 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] coordination.ZkSplitLogWorkerCoordination: successfully transitioned task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605566082047 to final state DONE cvp21,16201,1605569374704 2020-11-16 23:29:45,216 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] handler.WALSplitterHandler: Worker cvp21,16201,1605569374704 done with task org.apache.hadoop.hbase.coordination.ZkSplitLogWorkerCoordination$ZkSplitTaskDetails@37b3b41d in 768ms. Status = DONE 2020-11-16 23:29:45,295 INFO [SplitLogWorker-cvp21:16201] coordination.ZkSplitLogWorkerCoordination: worker cvp21,16201,1605569374704 acquired task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605566452906 2020-11-16 23:29:45,318 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.WALSplitter: Splitting WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566452906, size=2.2M (2301654 bytes) 2020-11-16 23:29:45,326 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] util.FSHDFSUtils: Recover lease on dfs file hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566452906 2020-11-16 23:29:45,328 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] util.FSHDFSUtils: Recovered lease, attempt=0 on file=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566452906 after 1ms 2020-11-16 23:29:45,347 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:45,349 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.WALSplitter: Open WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566452906 cost 31 ms 2020-11-16 23:29:45,426 INFO [Thread-64] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:45,473 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.OutputSink: 3 split writers finished; closing. 2020-11-16 23:29:45,578 INFO [split-log-closeStream-1] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/1a21a43d6ff9f33e59894a94c0613703/recovered.edits/0000000000260139441-cvp21%2C16201%2C1605560366412.1605566452906.temp to hdfs://mycluster/hbase/data/default/aeris_v2/1a21a43d6ff9f33e59894a94c0613703/recovered.edits/0000000000260139499 2020-11-16 23:29:45,579 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.WALSplitter: Processed 59 edits across 1 regions cost 228 ms; edits skipped=392; WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566452906, size=2.2M, length=2301654, corrupted=false, progress failed=false 2020-11-16 23:29:45,590 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] coordination.ZkSplitLogWorkerCoordination: successfully transitioned task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605566452906 to final state DONE cvp21,16201,1605569374704 2020-11-16 23:29:45,590 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] handler.WALSplitterHandler: Worker cvp21,16201,1605569374704 done with task org.apache.hadoop.hbase.coordination.ZkSplitLogWorkerCoordination$ZkSplitTaskDetails@9b4c89e in 295ms. Status = DONE 2020-11-16 23:29:45,811 INFO [SplitLogWorker-cvp21:16201] coordination.ZkSplitLogWorkerCoordination: worker cvp21,16201,1605569374704 acquired task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605566373966 2020-11-16 23:29:45,833 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.WALSplitter: Splitting WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566373966, size=94.2M (98772847 bytes) 2020-11-16 23:29:45,837 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] util.FSHDFSUtils: Recover lease on dfs file hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566373966 2020-11-16 23:29:45,838 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] util.FSHDFSUtils: Recovered lease, attempt=0 on file=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566373966 after 1ms 2020-11-16 23:29:45,857 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:45,859 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.WALSplitter: Open WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566373966 cost 26 ms 2020-11-16 23:29:45,891 INFO [Thread-66] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:46,575 INFO [SplitLogWorker-cvp21:16201] coordination.ZkSplitLogWorkerCoordination: worker cvp21,16201,1605569374704 acquired task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605567620621 2020-11-16 23:29:46,617 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.WALSplitter: Splitting WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567620621, size=0 (0 bytes) 2020-11-16 23:29:46,620 WARN [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.WALSplitter: File hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567620621 might be still open, length is 0 2020-11-16 23:29:46,620 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] util.FSHDFSUtils: Recover lease on dfs file hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567620621 2020-11-16 23:29:46,695 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] util.FSHDFSUtils: Failed to recover lease, attempt=0 on file=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567620621 after 74ms 2020-11-16 23:29:46,785 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.OutputSink: 3 split writers finished; closing. 2020-11-16 23:29:46,795 INFO [Thread-68] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:46,795 INFO [Thread-69] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:46,830 INFO [split-log-closeStream-1] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/__test-table_cvp21__/22d0b8a2f061eb1ac72b4ca27d122e8d/recovered.edits/0000000000000002009-cvp21%2C16201%2C1605560366412.1605566373966.temp to hdfs://mycluster/hbase/data/default/__test-table_cvp21__/22d0b8a2f061eb1ac72b4ca27d122e8d/recovered.edits/0000000000000002009 2020-11-16 23:29:46,831 INFO [split-log-closeStream-2] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/aeris_v2/1a21a43d6ff9f33e59894a94c0613703/recovered.edits/0000000000260136942-cvp21%2C16201%2C1605560366412.1605566373966.temp to hdfs://mycluster/hbase/data/default/aeris_v2/1a21a43d6ff9f33e59894a94c0613703/recovered.edits/0000000000260139442 2020-11-16 23:29:46,831 INFO [split-log-closeStream-3] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/__test-table_cvp22__/0bb85ddacc1f4a872bc5a7f3ebaa4878/recovered.edits/0000000000000002007-cvp21%2C16201%2C1605560366412.1605566373966.temp to hdfs://mycluster/hbase/data/default/__test-table_cvp22__/0bb85ddacc1f4a872bc5a7f3ebaa4878/recovered.edits/0000000000000002007 2020-11-16 23:29:46,832 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.WALSplitter: Processed 2503 edits across 3 regions cost 973 ms; edits skipped=14278; WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605566373966, size=94.2M, length=98772847, corrupted=false, progress failed=false 2020-11-16 23:29:46,837 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] coordination.ZkSplitLogWorkerCoordination: successfully transitioned task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605566373966 to final state DONE cvp21,16201,1605569374704 2020-11-16 23:29:46,837 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] handler.WALSplitterHandler: Worker cvp21,16201,1605569374704 done with task org.apache.hadoop.hbase.coordination.ZkSplitLogWorkerCoordination$ZkSplitTaskDetails@1aa515db in 1026ms. Status = DONE 2020-11-16 23:29:50,698 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] util.FSHDFSUtils: Recovered lease, attempt=1 on file=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567620621 after 4077ms 2020-11-16 23:29:50,704 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:29:50,708 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.WALSplitter: Open WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567620621 cost 4091 ms 2020-11-16 23:29:50,710 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.OutputSink: 3 split writers finished; closing. 2020-11-16 23:29:50,710 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] wal.WALSplitter: Processed 0 edits across 0 regions cost 2 ms; edits skipped=0; WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605567620621, size=0, length=0, corrupted=false, progress failed=false 2020-11-16 23:29:50,714 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] coordination.ZkSplitLogWorkerCoordination: successfully transitioned task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605567620621 to final state DONE cvp21,16201,1605569374704 2020-11-16 23:29:50,714 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-0] handler.WALSplitterHandler: Worker cvp21,16201,1605569374704 done with task org.apache.hadoop.hbase.coordination.ZkSplitLogWorkerCoordination$ZkSplitTaskDetails@3ef5c747 in 4138ms. Status = DONE 2020-11-16 23:30:38,130 INFO [ReadOnlyZKClient-cvp21:2181,cvp22:2181,cvp23:2181@0x438b9f1b] zookeeper.ZooKeeper: Session: 0x1000be3d4bf6c57 closed 2020-11-16 23:30:38,131 INFO [ReadOnlyZKClient-cvp21:2181,cvp22:2181,cvp23:2181@0x438b9f1b-EventThread] zookeeper.ClientCnxn: EventThread shut down for session: 0x1000be3d4bf6c57 2020-11-16 23:31:31,571 INFO [SplitLogWorker-cvp21:16201] coordination.ZkSplitLogWorkerCoordination: worker cvp21,16201,1605569374704 acquired task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605565244674 2020-11-16 23:31:31,600 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.WALSplitter: Splitting WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605565244674, size=128.3M (134511923 bytes) 2020-11-16 23:31:31,603 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] util.FSHDFSUtils: Recover lease on dfs file hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605565244674 2020-11-16 23:31:31,606 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] util.FSHDFSUtils: Recovered lease, attempt=0 on file=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605565244674 after 3ms 2020-11-16 23:31:31,614 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:31,624 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:31,626 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.WALSplitter: Open WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605565244674 cost 27 ms 2020-11-16 23:31:32,096 WARN [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1-Writer-1] wal.LogRecoveredEditsOutputSink: Found old edits file. It could be the result of a previous failed split attempt. Deleting hdfs://mycluster/hbase/data/default/__test-table_cvp21__/22d0b8a2f061eb1ac72b4ca27d122e8d/recovered.edits/0000000000000001997-cvp21%2C16201%2C1605560366412.1605565244674.temp, length=0 2020-11-16 23:31:32,098 WARN [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1-Writer-0] wal.LogRecoveredEditsOutputSink: Found old edits file. It could be the result of a previous failed split attempt. Deleting hdfs://mycluster/hbase/data/default/__test-table_cvp22__/0bb85ddacc1f4a872bc5a7f3ebaa4878/recovered.edits/0000000000000001995-cvp21%2C16201%2C1605560366412.1605565244674.temp, length=0 2020-11-16 23:31:32,838 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.OutputSink: 3 split writers finished; closing. 2020-11-16 23:31:32,849 INFO [Thread-77] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:32,849 INFO [Thread-76] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:32,890 INFO [split-log-closeStream-1] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/__test-table_cvp21__/22d0b8a2f061eb1ac72b4ca27d122e8d/recovered.edits/0000000000000001997-cvp21%2C16201%2C1605560366412.1605565244674.temp to hdfs://mycluster/hbase/data/default/__test-table_cvp21__/22d0b8a2f061eb1ac72b4ca27d122e8d/recovered.edits/0000000000000001999 2020-11-16 23:31:32,892 INFO [split-log-closeStream-2] wal.LogRecoveredEditsOutputSink: Rename hdfs://mycluster/hbase/data/default/__test-table_cvp22__/0bb85ddacc1f4a872bc5a7f3ebaa4878/recovered.edits/0000000000000001995-cvp21%2C16201%2C1605560366412.1605565244674.temp to hdfs://mycluster/hbase/data/default/__test-table_cvp22__/0bb85ddacc1f4a872bc5a7f3ebaa4878/recovered.edits/0000000000000001997 2020-11-16 23:31:32,892 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] wal.WALSplitter: Processed 6 edits across 2 regions cost 1266 ms; edits skipped=27890; WAL=hdfs://mycluster/hbase/WALs/cvp21,16201,1605560366412-splitting/cvp21%2C16201%2C1605560366412.1605565244674, size=128.3M, length=134511923, corrupted=false, progress failed=false 2020-11-16 23:31:32,897 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] coordination.ZkSplitLogWorkerCoordination: successfully transitioned task /hbase/splitWAL/WALs%2Fcvp21%2C16201%2C1605560366412-splitting%2Fcvp21%252C16201%252C1605560366412.1605565244674 to final state DONE cvp21,16201,1605569374704 2020-11-16 23:31:32,897 INFO [RS_LOG_REPLAY_OPS-regionserver/cvp21:16201-1] handler.WALSplitterHandler: Worker cvp21,16201,1605569374704 done with task org.apache.hadoop.hbase.coordination.ZkSplitLogWorkerCoordination$ZkSplitTaskDetails@1dc50f70 in 1326ms. Status = DONE 2020-11-16 23:31:34,314 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:34,347 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] handler.AssignRegionHandler: Open aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-16 23:31:34,357 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] handler.AssignRegionHandler: Open aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-16 23:31:34,363 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] handler.AssignRegionHandler: Open aeris_v2,,1598623754299.f30ff7a20d16bab8b2f9f9226e449a31. 2020-11-16 23:31:34,493 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint loaded, priority=536870911. 2020-11-16 23:31:34,493 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint loaded, priority=536870911. 2020-11-16 23:31:34,493 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint loaded, priority=536870911. 2020-11-16 23:31:34,499 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.AggregateImplementation loaded, priority=536870912. 2020-11-16 23:31:34,499 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.AggregateImplementation loaded, priority=536870912. 2020-11-16 23:31:34,499 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.AggregateImplementation loaded, priority=536870912. 2020-11-16 23:31:34,538 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-16 23:31:34,539 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-16 23:31:34,540 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-16 23:31:34,640 INFO [StoreOpener-bdaa31a63b7f3d06d9909b7560333e79-1] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false for family {NAME => 'e', VERSIONS => '43200', EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'false', CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'FAST_DIFF', TTL => '7776000 SECONDS (90 DAYS)', MIN_VERSIONS => '3', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'SNAPPY', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} with blockCache=LruBlockCache{blockCount=0, currentSize=1.38 MB, freeSize=1.83 GB, maxSize=1.83 GB, heapSize=1.38 MB, minSize=1.74 GB, minFactor=0.95, multiSize=892.05 MB, multiFactor=0.5, singleSize=446.02 MB, singleFactor=0.25} 2020-11-16 23:31:34,640 INFO [StoreOpener-f30ff7a20d16bab8b2f9f9226e449a31-1] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false for family {NAME => 'e', VERSIONS => '43200', EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'false', CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'FAST_DIFF', TTL => '7776000 SECONDS (90 DAYS)', MIN_VERSIONS => '3', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'SNAPPY', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} with blockCache=LruBlockCache{blockCount=0, currentSize=1.38 MB, freeSize=1.83 GB, maxSize=1.83 GB, heapSize=1.38 MB, minSize=1.74 GB, minFactor=0.95, multiSize=892.05 MB, multiFactor=0.5, singleSize=446.02 MB, singleFactor=0.25} 2020-11-16 23:31:34,640 INFO [StoreOpener-82421ce8c5c8b48c573baee83caa8305-1] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false for family {NAME => 'e', VERSIONS => '43200', EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'false', CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'FAST_DIFF', TTL => '7776000 SECONDS (90 DAYS)', MIN_VERSIONS => '3', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'SNAPPY', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} with blockCache=LruBlockCache{blockCount=0, currentSize=1.38 MB, freeSize=1.83 GB, maxSize=1.83 GB, heapSize=1.38 MB, minSize=1.74 GB, minFactor=0.95, multiSize=892.05 MB, multiFactor=0.5, singleSize=446.02 MB, singleFactor=0.25} 2020-11-16 23:31:34,655 INFO [StoreOpener-f30ff7a20d16bab8b2f9f9226e449a31-1] compactions.CompactionConfiguration: size [128 MB, 8.00 EB, 8.00 EB); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory 2020-11-16 23:31:34,655 INFO [StoreOpener-bdaa31a63b7f3d06d9909b7560333e79-1] compactions.CompactionConfiguration: size [128 MB, 8.00 EB, 8.00 EB); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory 2020-11-16 23:31:34,657 INFO [StoreOpener-82421ce8c5c8b48c573baee83caa8305-1] compactions.CompactionConfiguration: size [128 MB, 8.00 EB, 8.00 EB); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory 2020-11-16 23:31:34,707 INFO [StoreFileOpenerThread-e-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:34,749 WARN [StoreFileOpenerThread-e-1] retry.RetryInvocationHandler: A failover has occurred since the start of call #360 ClientNamenodeProtocolTranslatorPB.getBlockLocations over cvp22/192.168.2.22:9001 2020-11-16 23:31:34,763 INFO [StoreFileOpenerThread-e-1] compress.CodecPool: Got brand-new decompressor [.snappy] 2020-11-16 23:31:34,763 INFO [StoreFileOpenerThread-e-1] compress.CodecPool: Got brand-new decompressor [.snappy] 2020-11-16 23:31:34,767 INFO [StoreFileOpenerThread-e-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:34,860 INFO [StoreOpener-82421ce8c5c8b48c573baee83caa8305-1] regionserver.HStore: Store=e, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=FAST_DIFF, compression=SNAPPY 2020-11-16 23:31:34,860 INFO [StoreOpener-f30ff7a20d16bab8b2f9f9226e449a31-1] regionserver.HStore: Store=e, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=FAST_DIFF, compression=SNAPPY 2020-11-16 23:31:34,865 INFO [StoreOpener-bdaa31a63b7f3d06d9909b7560333e79-1] regionserver.HStore: Store=e, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=FAST_DIFF, compression=SNAPPY 2020-11-16 23:31:34,886 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/recovered.edits/0000000000251249906 2020-11-16 23:31:34,887 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/recovered.edits/0000000000259136918 2020-11-16 23:31:34,890 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/recovered.edits/0000000000290102943 2020-11-16 23:31:34,893 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:34,893 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:34,895 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:34,965 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/recovered.edits/0000000000259138796 2020-11-16 23:31:34,971 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:35,063 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/recovered.edits/0000000000251251268 2020-11-16 23:31:35,070 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:35,235 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter: Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2020-11-16 23:31:35,235 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter: Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.AggregateImplementation Metrics about HBase RegionObservers 2020-11-16 23:31:35,260 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/recovered.edits/0000000000290104117 2020-11-16 23:31:35,266 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:35,444 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/recovered.edits/0000000000290105293 2020-11-16 23:31:35,460 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:35,535 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/recovered.edits/0000000000259140444 2020-11-16 23:31:35,542 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:35,553 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/recovered.edits/0000000000290107153 2020-11-16 23:31:35,558 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:35,582 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/recovered.edits/0000000000251251382 2020-11-16 23:31:35,588 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:35,697 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=43.69 MB heapSize=52.50 MB 2020-11-16 23:31:35,764 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/recovered.edits/0000000000290108781 2020-11-16 23:31:35,830 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/recovered.edits/0000000000259142882 2020-11-16 23:31:35,836 INFO [Thread-82] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:35,839 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:35,898 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/recovered.edits/0000000000290111066 2020-11-16 23:31:35,904 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,027 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/recovered.edits/0000000000290114310 2020-11-16 23:31:36,033 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,039 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/recovered.edits/0000000000259145849 2020-11-16 23:31:36,044 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,120 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.DefaultStoreFlusher: Flushed memstore data size=43.69 MB at sequenceid=251251382 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/ab269a12f6e24512989d0f4d85bb6d09 2020-11-16 23:31:36,129 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,185 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/ab269a12f6e24512989d0f4d85bb6d09, entries=126049, sequenceid=251251382, filesize=2.6 M 2020-11-16 23:31:36,193 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Finished flush of dataSize ~43.69 MB/45809462, heapSize ~52.50 MB/55047176, currentSize=0 B/0 for 82421ce8c5c8b48c573baee83caa8305 in 497ms, sequenceid=251251382, compaction requested=true; wal=null 2020-11-16 23:31:36,228 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/recovered.edits/0000000000290114553 2020-11-16 23:31:36,239 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Opened 82421ce8c5c8b48c573baee83caa8305; next sequenceid=251251383 2020-11-16 23:31:36,261 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=30.61 MB heapSize=44.35 MB 2020-11-16 23:31:36,265 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegionServer: Post open deploy tasks for aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., openProcId=62, masterSystemTime=1605569494274 2020-11-16 23:31:36,277 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-16 23:31:36,278 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/29d41f91724f4e709d0342bc087b28b1, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/8b739aaef1b34583a4b5c40e7cd347df, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/ab269a12f6e24512989d0f4d85bb6d09] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=29.2 M 2020-11-16 23:31:36,315 INFO [Thread-85] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,351 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/recovered.edits/0000000000259146099 2020-11-16 23:31:36,357 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,377 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,379 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Flushing f30ff7a20d16bab8b2f9f9226e449a31 1/1 column families, dataSize=54.51 MB heapSize=65.84 MB 2020-11-16 23:31:36,384 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] handler.AssignRegionHandler: Opened aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-16 23:31:36,385 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] handler.AssignRegionHandler: Open __test-table_cvp23__,,1598623751841.bac2756ecc315bfd457fb71d81dd3eb5. 2020-11-16 23:31:36,386 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint loaded, priority=536870911. 2020-11-16 23:31:36,386 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.AggregateImplementation loaded, priority=536870912. 2020-11-16 23:31:36,398 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-16 23:31:36,400 INFO [StoreOpener-bac2756ecc315bfd457fb71d81dd3eb5-1] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false for family {NAME => '__status_family___', VERSIONS => '3', EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'false', CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'FAST_DIFF', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} with blockCache=LruBlockCache{blockCount=0, currentSize=1.38 MB, freeSize=1.83 GB, maxSize=1.83 GB, heapSize=1.38 MB, minSize=1.74 GB, minFactor=0.95, multiSize=892.05 MB, multiFactor=0.5, singleSize=446.02 MB, singleFactor=0.25} 2020-11-16 23:31:36,400 INFO [StoreOpener-bac2756ecc315bfd457fb71d81dd3eb5-1] compactions.CompactionConfiguration: size [128 MB, 8.00 EB, 8.00 EB); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory 2020-11-16 23:31:36,408 INFO [regionserver/cvp21:16201-shortCompactions-0] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-16 23:31:36,410 INFO [regionserver/cvp21:16201-shortCompactions-0] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-16 23:31:36,417 INFO [Thread-87] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,433 INFO [StoreFileOpenerThread-__status_family___-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,442 INFO [StoreOpener-bac2756ecc315bfd457fb71d81dd3eb5-1] regionserver.HStore: Store=__status_family___, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=FAST_DIFF, compression=NONE 2020-11-16 23:31:36,453 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/__test-table_cvp23__/bac2756ecc315bfd457fb71d81dd3eb5/recovered.edits/0000000000000002025 2020-11-16 23:31:36,470 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/__test-table_cvp23__/bac2756ecc315bfd457fb71d81dd3eb5/recovered.edits/0000000000000002027 2020-11-16 23:31:36,478 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/__test-table_cvp23__/bac2756ecc315bfd457fb71d81dd3eb5/recovered.edits/0000000000000002028 2020-11-16 23:31:36,484 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Flushing bac2756ecc315bfd457fb71d81dd3eb5 1/1 column families, dataSize=639 B heapSize=1.23 KB 2020-11-16 23:31:36,503 INFO [Thread-90] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,512 INFO [Thread-88] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,527 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.DefaultStoreFlusher: Flushed memstore data size=639 B at sequenceid=2028 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/__test-table_cvp23__/bac2756ecc315bfd457fb71d81dd3eb5/.tmp/__status_family___/f6a126b6539d46b89d35b08cf8a6b18d 2020-11-16 23:31:36,569 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/__test-table_cvp23__/bac2756ecc315bfd457fb71d81dd3eb5/__status_family___/f6a126b6539d46b89d35b08cf8a6b18d, entries=5, sequenceid=2028, filesize=5.2 K 2020-11-16 23:31:36,570 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Finished flush of dataSize ~639 B/639, heapSize ~1.21 KB/1240, currentSize=0 B/0 for bac2756ecc315bfd457fb71d81dd3eb5 in 86ms, sequenceid=2028, compaction requested=false; wal=null 2020-11-16 23:31:36,605 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Opened bac2756ecc315bfd457fb71d81dd3eb5; next sequenceid=2029 2020-11-16 23:31:36,609 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegionServer: Post open deploy tasks for __test-table_cvp23__,,1598623751841.bac2756ecc315bfd457fb71d81dd3eb5., openProcId=71, masterSystemTime=1605569494426 2020-11-16 23:31:36,722 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] handler.AssignRegionHandler: Opened __test-table_cvp23__,,1598623751841.bac2756ecc315bfd457fb71d81dd3eb5. 2020-11-16 23:31:36,722 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] handler.AssignRegionHandler: Open aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-16 23:31:36,723 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint loaded, priority=536870911. 2020-11-16 23:31:36,723 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.AggregateImplementation loaded, priority=536870912. 2020-11-16 23:31:36,733 INFO [StoreOpener-0469ce862509756add4a8cdb00bc465c-1] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false for family {NAME => 'e', VERSIONS => '43200', EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'false', CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'FAST_DIFF', TTL => '7776000 SECONDS (90 DAYS)', MIN_VERSIONS => '3', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'SNAPPY', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} with blockCache=LruBlockCache{blockCount=0, currentSize=1.38 MB, freeSize=1.83 GB, maxSize=1.83 GB, heapSize=1.38 MB, minSize=1.74 GB, minFactor=0.95, multiSize=892.05 MB, multiFactor=0.5, singleSize=446.02 MB, singleFactor=0.25} 2020-11-16 23:31:36,734 INFO [StoreOpener-0469ce862509756add4a8cdb00bc465c-1] compactions.CompactionConfiguration: size [128 MB, 8.00 EB, 8.00 EB); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory 2020-11-16 23:31:36,745 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.DefaultStoreFlusher: Flushed memstore data size=30.61 MB at sequenceid=290114553 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/4d449c6b2eae4a758abd0d15976f713a 2020-11-16 23:31:36,771 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,776 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/4d449c6b2eae4a758abd0d15976f713a, entries=191115, sequenceid=290114553, filesize=2.9 M 2020-11-16 23:31:36,777 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Finished flush of dataSize ~30.61 MB/32100285, heapSize ~44.35 MB/46502056, currentSize=0 B/0 for bdaa31a63b7f3d06d9909b7560333e79 in 516ms, sequenceid=290114553, compaction requested=true; wal=null 2020-11-16 23:31:36,814 INFO [StoreOpener-0469ce862509756add4a8cdb00bc465c-1] regionserver.HStore: Store=e, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=FAST_DIFF, compression=SNAPPY 2020-11-16 23:31:36,824 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Opened bdaa31a63b7f3d06d9909b7560333e79; next sequenceid=290114554 2020-11-16 23:31:36,827 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegionServer: Post open deploy tasks for aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., openProcId=64, masterSystemTime=1605569494274 2020-11-16 23:31:36,830 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-16 23:31:36,830 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/b29d28554e8b44809b61171dc4f5aec1, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/286a0b04729e49878710b5d9ff0e0a6d, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/4d449c6b2eae4a758abd0d15976f713a] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp, totalSize=31.3 M 2020-11-16 23:31:36,832 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/recovered.edits/0000000000270768787 2020-11-16 23:31:36,838 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,871 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,879 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,889 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/recovered.edits/0000000000270769986 2020-11-16 23:31:36,897 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,921 INFO [Thread-95] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:36,981 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] handler.AssignRegionHandler: Opened aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-16 23:31:36,981 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.DefaultStoreFlusher: Flushed memstore data size=54.51 MB at sequenceid=259146099 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/.tmp/e/5f31a50878094f69942a0a99c5f1d4ef 2020-11-16 23:31:36,994 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:37,068 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/e/5f31a50878094f69942a0a99c5f1d4ef, entries=156816, sequenceid=259146099, filesize=4.3 M 2020-11-16 23:31:37,069 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Finished flush of dataSize ~54.51 MB/57160550, heapSize ~65.84 MB/69035360, currentSize=0 B/0 for f30ff7a20d16bab8b2f9f9226e449a31 in 691ms, sequenceid=259146099, compaction requested=true; wal=null 2020-11-16 23:31:37,109 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Opened f30ff7a20d16bab8b2f9f9226e449a31; next sequenceid=259146100 2020-11-16 23:31:37,112 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.HRegionServer: Post open deploy tasks for aeris_v2,,1598623754299.f30ff7a20d16bab8b2f9f9226e449a31., openProcId=66, masterSystemTime=1605569494274 2020-11-16 23:31:37,224 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] handler.AssignRegionHandler: Opened aeris_v2,,1598623754299.f30ff7a20d16bab8b2f9f9226e449a31. 2020-11-16 23:31:37,363 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/recovered.edits/0000000000270771819 2020-11-16 23:31:37,952 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/recovered.edits/0000000000270774723 2020-11-16 23:31:37,959 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:38,178 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/recovered.edits/0000000000270777050 2020-11-16 23:31:38,183 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:38,347 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/recovered.edits/0000000000270779940 2020-11-16 23:31:38,362 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:38,668 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Replaying edits from hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/recovered.edits/0000000000270780188 2020-11-16 23:31:38,673 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:38,704 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=80.36 MB heapSize=94.88 MB 2020-11-16 23:31:38,724 INFO [Thread-98] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:39,186 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.DefaultStoreFlusher: Flushed memstore data size=80.36 MB at sequenceid=270780188 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/adb064193a994aecbe550670ea4e3d78 2020-11-16 23:31:39,212 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/adb064193a994aecbe550670ea4e3d78, entries=198971, sequenceid=270780188, filesize=6.2 M 2020-11-16 23:31:39,213 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Finished flush of dataSize ~80.36 MB/84266271, heapSize ~94.88 MB/99491360, currentSize=0 B/0 for 0469ce862509756add4a8cdb00bc465c in 510ms, sequenceid=270780188, compaction requested=true; wal=null 2020-11-16 23:31:39,252 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Opened 0469ce862509756add4a8cdb00bc465c; next sequenceid=270780189 2020-11-16 23:31:39,255 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegionServer: Post open deploy tasks for aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., openProcId=74, masterSystemTime=1605569494426 2020-11-16 23:31:39,371 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] handler.AssignRegionHandler: Opened aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-16 23:31:47,259 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: bdaa31a63b7f3d06d9909b7560333e79#e#compaction#5 average throughput is 31.16 MB/second, slept 6 time(s) and total slept time is 6371 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-16 23:31:47,393 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of bdaa31a63b7f3d06d9909b7560333e79 into d5b40a92e68849e6ba96b1cb743ab459(size=31.5 M), total size for store is 31.7 G. This selection was in queue for 0sec, and took 10sec to execute. 2020-11-16 23:31:47,393 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., storeName=e, priority=11, startTime=1605569496827; duration=10sec 2020-11-16 23:31:47,397 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,,1598623754299.f30ff7a20d16bab8b2f9f9226e449a31. 2020-11-16 23:31:47,398 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/e/43e79fbb5c8c4a3aac6c14509f822c78, hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/e/db98bba1482845c2bf6848d8aff8d112, hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/e/5f31a50878094f69942a0a99c5f1d4ef] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/.tmp, totalSize=39.0 M 2020-11-16 23:31:47,429 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:47,433 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:47,459 INFO [Thread-102] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:51,081 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#3 average throughput is 31.23 MB/second, slept 9 time(s) and total slept time is 10741 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-16 23:31:51,198 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 0576e39e8a0a45b6ae90175d859bf72b(size=29.2 M), total size for store is 10.8 G. This selection was in queue for 0sec, and took 14sec to execute. 2020-11-16 23:31:51,198 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605569496269; duration=14sec 2020-11-16 23:31:57,343 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: f30ff7a20d16bab8b2f9f9226e449a31#e#compaction#7 average throughput is 47.21 MB/second, slept 9 time(s) and total slept time is 6340 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-16 23:31:57,465 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of f30ff7a20d16bab8b2f9f9226e449a31 into 4941ce187a334debb688916818f4b5e8(size=39.4 M), total size for store is 54.9 G. This selection was in queue for 0sec, and took 10sec to execute. 2020-11-16 23:31:57,465 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,,1598623754299.f30ff7a20d16bab8b2f9f9226e449a31., storeName=e, priority=10, startTime=1605569497112; duration=10sec 2020-11-16 23:31:59,121 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:31:59,125 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner -6324287692082348585 2020-11-16 23:31:59,128 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] compress.CodecPool: Got brand-new decompressor [.snappy] 2020-11-16 23:31:59,144 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:32:09,174 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:32:09,174 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:32:09,178 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner -6324287692082348595 2020-11-16 23:32:09,179 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:32:20,326 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:32:34,795 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:32:35,962 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:32:35,962 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:32:36,004 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:32:36,005 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:32:36,005 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:32:36,005 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:32:36,005 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:33:06,996 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:33:53,455 INFO [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:33:53,458 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] handler.AssignRegionHandler: Open aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-16 23:33:53,458 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint loaded, priority=536870911. 2020-11-16 23:33:53,459 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.AggregateImplementation loaded, priority=536870912. 2020-11-16 23:33:53,469 INFO [StoreOpener-5e5f05205d7e40b0141cd977dd009ba1-1] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false for family {NAME => 'e', VERSIONS => '43200', EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'false', CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'FAST_DIFF', TTL => '7776000 SECONDS (90 DAYS)', MIN_VERSIONS => '3', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'SNAPPY', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} with blockCache=LruBlockCache{blockCount=118, currentSize=10.00 MB, freeSize=1.82 GB, maxSize=1.83 GB, heapSize=10.00 MB, minSize=1.74 GB, minFactor=0.95, multiSize=892.05 MB, multiFactor=0.5, singleSize=446.02 MB, singleFactor=0.25} 2020-11-16 23:33:53,469 INFO [StoreOpener-5e5f05205d7e40b0141cd977dd009ba1-1] compactions.CompactionConfiguration: size [128 MB, 8.00 EB, 8.00 EB); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory 2020-11-16 23:33:53,549 INFO [StoreOpener-5e5f05205d7e40b0141cd977dd009ba1-1] regionserver.HStore: Store=e, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=FAST_DIFF, compression=SNAPPY 2020-11-16 23:33:53,564 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Opened 5e5f05205d7e40b0141cd977dd009ba1; next sequenceid=286196966 2020-11-16 23:33:53,566 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegionServer: Post open deploy tasks for aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., openProcId=77, masterSystemTime=1605569633435 2020-11-16 23:33:53,678 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] handler.AssignRegionHandler: Opened aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-16 23:33:54,549 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] handler.AssignRegionHandler: Open aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-16 23:33:54,549 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint loaded, priority=536870911. 2020-11-16 23:33:54,550 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.AggregateImplementation loaded, priority=536870912. 2020-11-16 23:33:54,562 INFO [StoreOpener-73aae5715542e1a5370f7540d2117abc-1] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false for family {NAME => 'e', VERSIONS => '43200', EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'false', CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'FAST_DIFF', TTL => '7776000 SECONDS (90 DAYS)', MIN_VERSIONS => '3', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'SNAPPY', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} with blockCache=LruBlockCache{blockCount=118, currentSize=10.00 MB, freeSize=1.82 GB, maxSize=1.83 GB, heapSize=10.00 MB, minSize=1.74 GB, minFactor=0.95, multiSize=892.05 MB, multiFactor=0.5, singleSize=446.02 MB, singleFactor=0.25} 2020-11-16 23:33:54,562 INFO [StoreOpener-73aae5715542e1a5370f7540d2117abc-1] compactions.CompactionConfiguration: size [128 MB, 8.00 EB, 8.00 EB); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory 2020-11-16 23:33:54,640 INFO [StoreOpener-73aae5715542e1a5370f7540d2117abc-1] regionserver.HStore: Store=e, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=FAST_DIFF, compression=SNAPPY 2020-11-16 23:33:54,657 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Opened 73aae5715542e1a5370f7540d2117abc; next sequenceid=260431502 2020-11-16 23:33:54,660 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.HRegionServer: Post open deploy tasks for aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., openProcId=80, masterSystemTime=1605569634540 2020-11-16 23:33:54,662 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-16 23:33:54,662 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/3c2b05ca2fa14478a884d64797d17bf9, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/f9c9f679570e4f49a7edc0d64f7fed59, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/eb04d8b09c3d41d2900fb13ad54b5e41] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=72.6 M 2020-11-16 23:33:54,692 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:33:54,694 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:33:54,728 INFO [Thread-109] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:33:54,772 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] handler.AssignRegionHandler: Opened aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-16 23:33:58,630 INFO [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:33:58,632 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] handler.AssignRegionHandler: Open aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-16 23:33:58,633 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint loaded, priority=536870911. 2020-11-16 23:33:58,633 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.AggregateImplementation loaded, priority=536870912. 2020-11-16 23:33:58,642 INFO [StoreOpener-e5dc5c620be19b02666a19c280ff582e-1] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false for family {NAME => 'e', VERSIONS => '43200', EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'false', CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'FAST_DIFF', TTL => '7776000 SECONDS (90 DAYS)', MIN_VERSIONS => '3', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'SNAPPY', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} with blockCache=LruBlockCache{blockCount=118, currentSize=10.00 MB, freeSize=1.82 GB, maxSize=1.83 GB, heapSize=10.00 MB, minSize=1.74 GB, minFactor=0.95, multiSize=892.05 MB, multiFactor=0.5, singleSize=446.02 MB, singleFactor=0.25} 2020-11-16 23:33:58,642 INFO [StoreOpener-e5dc5c620be19b02666a19c280ff582e-1] compactions.CompactionConfiguration: size [128 MB, 8.00 EB, 8.00 EB); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory 2020-11-16 23:33:58,733 INFO [StoreOpener-e5dc5c620be19b02666a19c280ff582e-1] regionserver.HStore: Store=e, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=FAST_DIFF, compression=SNAPPY 2020-11-16 23:33:58,748 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Opened e5dc5c620be19b02666a19c280ff582e; next sequenceid=261955493 2020-11-16 23:33:58,751 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] regionserver.HRegionServer: Post open deploy tasks for aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., openProcId=89, masterSystemTime=1605569638624 2020-11-16 23:33:58,862 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-0] handler.AssignRegionHandler: Opened aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-16 23:34:00,565 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] handler.AssignRegionHandler: Open aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-16 23:34:00,566 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint loaded, priority=536870911. 2020-11-16 23:34:00,566 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.AggregateImplementation loaded, priority=536870912. 2020-11-16 23:34:00,575 INFO [StoreOpener-fc93f6dcc564341dcc983bf30971f73d-1] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false for family {NAME => 'e', VERSIONS => '43200', EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'false', CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'FAST_DIFF', TTL => '7776000 SECONDS (90 DAYS)', MIN_VERSIONS => '3', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'SNAPPY', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} with blockCache=LruBlockCache{blockCount=118, currentSize=10.00 MB, freeSize=1.82 GB, maxSize=1.83 GB, heapSize=10.00 MB, minSize=1.74 GB, minFactor=0.95, multiSize=892.05 MB, multiFactor=0.5, singleSize=446.02 MB, singleFactor=0.25} 2020-11-16 23:34:00,575 INFO [StoreOpener-fc93f6dcc564341dcc983bf30971f73d-1] compactions.CompactionConfiguration: size [128 MB, 8.00 EB, 8.00 EB); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory 2020-11-16 23:34:00,647 INFO [StoreOpener-fc93f6dcc564341dcc983bf30971f73d-1] regionserver.HStore: Store=e, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=FAST_DIFF, compression=SNAPPY 2020-11-16 23:34:00,663 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Opened fc93f6dcc564341dcc983bf30971f73d; next sequenceid=272217933 2020-11-16 23:34:00,666 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] regionserver.HRegionServer: Post open deploy tasks for aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., openProcId=92, masterSystemTime=1605569640558 2020-11-16 23:34:00,778 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-1] handler.AssignRegionHandler: Opened aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-16 23:34:01,328 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:34:01,650 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] handler.AssignRegionHandler: Open aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-16 23:34:01,651 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint loaded, priority=536870911. 2020-11-16 23:34:01,651 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] coprocessor.CoprocessorHost: System coprocessor org.apache.hadoop.hbase.coprocessor.AggregateImplementation loaded, priority=536870912. 2020-11-16 23:34:01,661 INFO [StoreOpener-06f7ca9534710e37d90d9f8345864259-1] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false for family {NAME => 'e', VERSIONS => '43200', EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'false', CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'FAST_DIFF', TTL => '7776000 SECONDS (90 DAYS)', MIN_VERSIONS => '3', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'SNAPPY', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} with blockCache=LruBlockCache{blockCount=136, currentSize=11.46 MB, freeSize=1.82 GB, maxSize=1.83 GB, heapSize=11.46 MB, minSize=1.74 GB, minFactor=0.95, multiSize=892.05 MB, multiFactor=0.5, singleSize=446.02 MB, singleFactor=0.25} 2020-11-16 23:34:01,661 INFO [StoreOpener-06f7ca9534710e37d90d9f8345864259-1] compactions.CompactionConfiguration: size [128 MB, 8.00 EB, 8.00 EB); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory 2020-11-16 23:34:01,788 INFO [StoreOpener-06f7ca9534710e37d90d9f8345864259-1] regionserver.HStore: Store=e, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=FAST_DIFF, compression=SNAPPY 2020-11-16 23:34:01,815 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Opened 06f7ca9534710e37d90d9f8345864259; next sequenceid=258585318 2020-11-16 23:34:01,817 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] regionserver.HRegionServer: Post open deploy tasks for aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., openProcId=95, masterSystemTime=1605569641640 2020-11-16 23:34:01,939 INFO [RS_OPEN_REGION-regionserver/cvp21:16201-2] handler.AssignRegionHandler: Opened aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-16 23:34:02,317 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-0] handler.UnassignRegionHandler: Close f30ff7a20d16bab8b2f9f9226e449a31 2020-11-16 23:34:02,319 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:34:02,319 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:34:02,326 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:34:02,326 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Flushing f30ff7a20d16bab8b2f9f9226e449a31 1/1 column families, dataSize=409.23 KB heapSize=687.48 KB 2020-11-16 23:34:02,364 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] compress.CodecPool: Got brand-new decompressor [.snappy] 2020-11-16 23:34:02,475 INFO [Thread-112] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:34:02,516 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-16 23:34:02,549 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-0] regionserver.DefaultStoreFlusher: Flushed memstore data size=409.23 KB at sequenceid=259146880 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/.tmp/e/3496010a9e25425a9257394816054f64 2020-11-16 23:34:02,563 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:34:02,564 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:34:02,575 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:34:02,664 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/f30ff7a20d16bab8b2f9f9226e449a31/e/3496010a9e25425a9257394816054f64, entries=3282, sequenceid=259146880, filesize=59.5 K 2020-11-16 23:34:02,675 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Finished flush of dataSize ~409.23 KB/419052, heapSize ~687.46 KB/703960, currentSize=0 B/0 for f30ff7a20d16bab8b2f9f9226e449a31 in 349ms, sequenceid=259146880, compaction requested=true 2020-11-16 23:34:02,879 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Closed aeris_v2,,1598623754299.f30ff7a20d16bab8b2f9f9226e449a31. 2020-11-16 23:34:02,879 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-0] regionserver.HRegionServer: Adding f30ff7a20d16bab8b2f9f9226e449a31 move to cvp23,16201,1605473602599 record at close sequenceid=259146880 2020-11-16 23:34:03,024 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-0] handler.UnassignRegionHandler: Closed f30ff7a20d16bab8b2f9f9226e449a31 2020-11-16 23:34:04,927 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:34:05,461 INFO [RS-EventLoopGroup-1-13] compress.CodecPool: Got brand-new decompressor [.snappy] 2020-11-16 23:34:05,462 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] compress.CodecPool: Got brand-new decompressor [.snappy] 2020-11-16 23:34:05,785 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] compress.CodecPool: Got brand-new decompressor [.snappy] 2020-11-16 23:34:06,438 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-16 23:34:06,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-16 23:34:06,683 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-16 23:34:06,683 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-16 23:34:06,686 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-16 23:34:07,371 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner -6324287692082348742 2020-11-16 23:34:08,420 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] compress.CodecPool: Got brand-new decompressor [.snappy] 2020-11-16 23:34:09,819 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-16 23:34:09,819 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-16 23:34:13,784 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#8 average throughput is 37.55 MB/second, slept 10 time(s) and total slept time is 3003 ms. 0 active operations remaining, total limit is 53.85 MB/second 2020-11-16 23:34:14,090 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into d0bbb11af36c43cab73875bf442c8632(size=72.6 M), total size for store is 12.3 G. This selection was in queue for 0sec, and took 19sec to execute. 2020-11-16 23:34:14,090 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605569634660; duration=19sec 2020-11-16 23:34:15,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:34:31,597 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:34:31,597 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:34:31,598 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:34:31,598 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:34:31,599 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=655.87 MB, freeSize=1.19 GB, max=1.83 GB, blockCount=13059, accesses=458956, hits=412102, hitRatio=89.79%, , cachingAccesses=426841, cachingHits=412101, cachingHitsRatio=96.55%, evictions=29, evicted=0, evictedPerRun=0.0 2020-11-16 23:34:38,665 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 93764 ns. 2020-11-16 23:34:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 40514 ns. 2020-11-16 23:34:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 48900 ns. 2020-11-16 23:34:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 45715 ns. 2020-11-16 23:35:16,097 INFO [qtp368802354-78] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-16 23:35:18,458 INFO [qtp368802354-316] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-16 23:35:19,592 INFO [qtp368802354-78] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-16 23:35:20,501 INFO [qtp368802354-71] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-16 23:35:21,171 INFO [qtp368802354-316] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-16 23:35:38,069 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:35:38,429 INFO [qtp368802354-71] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-16 23:35:39,386 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 7231112 ns. 2020-11-16 23:35:39,393 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 133490 ns. 2020-11-16 23:35:39,393 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 58988 ns. 2020-11-16 23:35:40,687 INFO [qtp368802354-317] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-16 23:35:42,324 INFO [qtp368802354-78] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-16 23:35:43,969 INFO [qtp368802354-78] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-16 23:35:45,453 INFO [qtp368802354-72] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-16 23:35:50,085 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:35:50,088 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:36:16,835 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569383863 with entries=49571, filesize=128.18 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569776761 2020-11-16 23:36:32,968 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569776761 with entries=8916, filesize=129.22 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569792885 2020-11-16 23:36:57,628 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:36:59,671 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569792885 with entries=10439, filesize=128.86 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569819634 2020-11-16 23:37:16,365 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.29 MB heapSize=128.42 MB 2020-11-16 23:37:16,431 INFO [Thread-124] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:37:17,343 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.29 MB at sequenceid=261965081 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/d28ba1ffe3c14fce94a8926e2971e813 2020-11-16 23:37:17,373 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d28ba1ffe3c14fce94a8926e2971e813, entries=379822, sequenceid=261965081, filesize=7.3 M 2020-11-16 23:37:17,375 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.29 MB/105158682, heapSize ~128.42 MB/134654504, currentSize=771.18 KB/789690 for e5dc5c620be19b02666a19c280ff582e in 1011ms, sequenceid=261965081, compaction requested=true 2020-11-16 23:37:50,164 INFO [qtp368802354-72] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-16 23:37:51,158 INFO [qtp368802354-317] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-16 23:37:53,037 INFO [qtp368802354-78] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-16 23:38:12,166 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=106.50 MB heapSize=128.00 MB 2020-11-16 23:38:12,196 INFO [Thread-129] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:38:12,884 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.50 MB at sequenceid=270802786 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/1bbdaa9e556e479cb2505a1a6077c9ca 2020-11-16 23:38:12,917 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/1bbdaa9e556e479cb2505a1a6077c9ca, entries=265116, sequenceid=270802786, filesize=8.0 M 2020-11-16 23:38:12,919 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.50 MB/111668309, heapSize ~128.00 MB/134218096, currentSize=26.30 KB/26930 for 0469ce862509756add4a8cdb00bc465c in 753ms, sequenceid=270802786, compaction requested=true 2020-11-16 23:38:15,191 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569819634 with entries=25476, filesize=128.36 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569895159 2020-11-16 23:38:36,510 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=106.67 MB heapSize=128.71 MB 2020-11-16 23:38:36,551 INFO [Thread-132] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:38:37,257 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.67 MB at sequenceid=261968853 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/653d3b56f04b4a72b1e9c08c42201ccd 2020-11-16 23:38:37,302 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/653d3b56f04b4a72b1e9c08c42201ccd, entries=304060, sequenceid=261968853, filesize=6.8 M 2020-11-16 23:38:37,321 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.67 MB/111856346, heapSize ~128.71 MB/134958800, currentSize=1.83 MB/1918928 for e5dc5c620be19b02666a19c280ff582e in 810ms, sequenceid=261968853, compaction requested=true 2020-11-16 23:38:37,322 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-16 23:38:37,322 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6ce9b940d20b42f2a4b5a252f432f8ff, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d28ba1ffe3c14fce94a8926e2971e813, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/653d3b56f04b4a72b1e9c08c42201ccd] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=36.4 M 2020-11-16 23:38:37,391 INFO [Thread-134] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:38:45,305 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#13 average throughput is 62.90 MB/second, slept 9 time(s) and total slept time is 3112 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-16 23:38:45,379 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 3e43307072124622995982c0de3aabee(size=36.4 M), total size for store is 48.2 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-16 23:38:45,379 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605569917321; duration=8sec 2020-11-16 23:39:00,618 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:39:12,614 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569895159 with entries=19191, filesize=128.40 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569952586 2020-11-16 23:39:17,091 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.23 GB, freeSize=620.82 MB, max=1.83 GB, blockCount=33382, accesses=1544268, hits=1465334, hitRatio=94.89%, , cachingAccesses=1504052, cachingHits=1465301, cachingHitsRatio=97.42%, evictions=59, evicted=0, evictedPerRun=0.0 2020-11-16 23:39:36,693 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:39:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 114022 ns. 2020-11-16 23:39:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 36273 ns. 2020-11-16 23:39:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 42509 ns. 2020-11-16 23:39:40,178 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:40:04,948 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.31 MB heapSize=128.00 MB 2020-11-16 23:40:04,989 INFO [Thread-139] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:40:05,668 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.31 MB at sequenceid=261972777 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/799c3d025b234ba29f3b2d411d057ec3 2020-11-16 23:40:05,677 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:40:05,697 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/799c3d025b234ba29f3b2d411d057ec3, entries=341726, sequenceid=261972777, filesize=6.9 M 2020-11-16 23:40:05,698 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.31 MB/108332988, heapSize ~128.00 MB/134218184, currentSize=20.88 KB/21376 for e5dc5c620be19b02666a19c280ff582e in 750ms, sequenceid=261972777, compaction requested=true 2020-11-16 23:40:39,327 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 57814 ns. 2020-11-16 23:40:41,684 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569952586 with entries=29336, filesize=128.67 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570041657 2020-11-16 23:40:45,928 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:41:39,045 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 94700 ns. 2020-11-16 23:41:39,046 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 1536220 ns. 2020-11-16 23:41:39,046 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 88359 ns. 2020-11-16 23:42:33,013 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570041657 with entries=37192, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570152965 2020-11-16 23:43:01,077 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=109.75 MB heapSize=128.36 MB 2020-11-16 23:43:01,146 INFO [Thread-148] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:43:01,580 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:43:01,868 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=109.75 MB at sequenceid=272230490 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/e317723427fb41658b89ea2705fe6931 2020-11-16 23:43:01,876 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:43:01,913 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/e317723427fb41658b89ea2705fe6931, entries=251068, sequenceid=272230490, filesize=8.2 M 2020-11-16 23:43:01,915 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~109.75 MB/115082224, heapSize ~128.36 MB/134598568, currentSize=664.58 KB/680534 for fc93f6dcc564341dcc983bf30971f73d in 838ms, sequenceid=272230490, compaction requested=true 2020-11-16 23:43:19,088 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.37 MB heapSize=128.05 MB 2020-11-16 23:43:19,116 INFO [Thread-150] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:43:19,931 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.37 MB at sequenceid=261981440 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/01476ff79ff94feca4031e11b326bf63 2020-11-16 23:43:19,939 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:43:19,960 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/01476ff79ff94feca4031e11b326bf63, entries=399329, sequenceid=261981440, filesize=7.6 M 2020-11-16 23:43:19,962 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.37 MB/104197414, heapSize ~128.05 MB/134274736, currentSize=603.98 KB/618476 for e5dc5c620be19b02666a19c280ff582e in 874ms, sequenceid=261981440, compaction requested=true 2020-11-16 23:43:19,963 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-16 23:43:19,963 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3e43307072124622995982c0de3aabee, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/799c3d025b234ba29f3b2d411d057ec3, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/01476ff79ff94feca4031e11b326bf63] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=50.9 M 2020-11-16 23:43:19,997 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:43:20,000 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:43:20,029 INFO [Thread-152] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:43:32,419 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#17 average throughput is 56.57 MB/second, slept 12 time(s) and total slept time is 4610 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-16 23:43:32,485 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:43:32,508 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 5b5e55c0f268423691f83909bf54655d(size=51.4 M), total size for store is 48.3 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-16 23:43:32,509 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605570199962; duration=12sec 2020-11-16 23:44:17,474 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570152965 with entries=35906, filesize=128.86 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570257439 2020-11-16 23:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.50 GB, freeSize=342.10 MB, max=1.83 GB, blockCount=39218, accesses=1627120, hits=1530926, hitRatio=94.09%, , cachingAccesses=1575515, cachingHits=1530887, cachingHitsRatio=97.17%, evictions=89, evicted=0, evictedPerRun=0.0 2020-11-16 23:44:39,403 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 105925 ns. 2020-11-16 23:44:39,403 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 35208 ns. 2020-11-16 23:44:39,403 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 37578 ns. 2020-11-16 23:44:55,126 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:45:11,793 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:45:59,773 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:46:15,615 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.99 MB heapSize=128.29 MB 2020-11-16 23:46:15,660 INFO [Thread-160] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:46:16,783 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.99 MB at sequenceid=260469204 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/9d68b5a261d14c478646372105eeaf5a 2020-11-16 23:46:16,792 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:46:16,816 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/9d68b5a261d14c478646372105eeaf5a, entries=575048, sequenceid=260469204, filesize=7.4 M 2020-11-16 23:46:16,818 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.99 MB/90162091, heapSize ~128.29 MB/134518120, currentSize=135.24 KB/138490 for 73aae5715542e1a5370f7540d2117abc in 1203ms, sequenceid=260469204, compaction requested=true 2020-11-16 23:46:19,751 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570257439 with entries=52840, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570379725 2020-11-16 23:46:20,385 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:46:33,346 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:46:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 73126 ns. 2020-11-16 23:46:42,408 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:46:53,038 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.63 MB heapSize=128.27 MB 2020-11-16 23:46:53,081 INFO [Thread-165] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:46:53,887 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.63 MB at sequenceid=261993426 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/b00b3bcf1f6745b0b873f698cea1b77f 2020-11-16 23:46:53,897 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:46:53,921 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b00b3bcf1f6745b0b873f698cea1b77f, entries=383871, sequenceid=261993426, filesize=8.1 M 2020-11-16 23:46:53,923 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.63 MB/105513934, heapSize ~128.27 MB/134497680, currentSize=962.23 KB/985320 for e5dc5c620be19b02666a19c280ff582e in 885ms, sequenceid=261993426, compaction requested=true 2020-11-16 23:47:04,359 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:47:12,292 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:47:25,800 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:47:38,010 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570379725 with entries=30809, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570457977 2020-11-16 23:47:38,835 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 76831 ns. 2020-11-16 23:47:38,836 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 307995 ns. 2020-11-16 23:47:38,836 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 65614 ns. 2020-11-16 23:47:41,913 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:47:49,636 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:48:18,025 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=94.80 MB heapSize=128.29 MB 2020-11-16 23:48:18,072 INFO [Thread-170] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:48:19,078 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=94.80 MB at sequenceid=258628742 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/0a9f1278b4ae406db3a593940b2883cd 2020-11-16 23:48:19,084 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:48:19,105 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/0a9f1278b4ae406db3a593940b2883cd, entries=463078, sequenceid=258628742, filesize=7.1 M 2020-11-16 23:48:19,108 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~94.80 MB/99403443, heapSize ~128.29 MB/134517080, currentSize=38.08 KB/38994 for 06f7ca9534710e37d90d9f8345864259 in 1083ms, sequenceid=258628742, compaction requested=true 2020-11-16 23:48:29,751 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:48:40,961 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:48:50,392 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:49:33,544 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570457977 with entries=43643, filesize=128.25 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570573506 2020-11-16 23:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.50 GB, freeSize=338.33 MB, max=1.83 GB, blockCount=39303, accesses=1632794, hits=1536515, hitRatio=94.10%, , cachingAccesses=1581189, cachingHits=1536476, cachingHitsRatio=97.17%, evictions=119, evicted=0, evictedPerRun=0.0 2020-11-16 23:49:48,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 41701 ns. 2020-11-16 23:49:48,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 46650 ns. 2020-11-16 23:50:00,072 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:50:05,583 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=102.87 MB heapSize=128.00 MB 2020-11-16 23:50:05,647 INFO [Thread-176] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:50:06,329 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:50:06,411 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.87 MB at sequenceid=251276281 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/d1cce499794a401f803246348e2501bd 2020-11-16 23:50:06,446 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d1cce499794a401f803246348e2501bd, entries=351713, sequenceid=251276281, filesize=6.8 M 2020-11-16 23:50:06,448 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.87 MB/107865346, heapSize ~128.00 MB/134220528, currentSize=298.98 KB/306155 for 82421ce8c5c8b48c573baee83caa8305 in 866ms, sequenceid=251276281, compaction requested=true 2020-11-16 23:50:19,750 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.84 MB heapSize=128.21 MB 2020-11-16 23:50:19,804 INFO [Thread-178] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:50:20,642 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.84 MB at sequenceid=262004006 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/08aae0eb756848dcaa07ad071c266504 2020-11-16 23:50:20,651 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:50:20,698 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/08aae0eb756848dcaa07ad071c266504, entries=365863, sequenceid=262004006, filesize=8.2 M 2020-11-16 23:50:20,700 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.84 MB/106788886, heapSize ~128.21 MB/134434744, currentSize=598.32 KB/612680 for e5dc5c620be19b02666a19c280ff582e in 950ms, sequenceid=262004006, compaction requested=true 2020-11-16 23:50:20,701 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-16 23:50:20,701 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5b5e55c0f268423691f83909bf54655d, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b00b3bcf1f6745b0b873f698cea1b77f, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/08aae0eb756848dcaa07ad071c266504] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=67.6 M 2020-11-16 23:50:20,758 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:50:20,763 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:50:20,807 INFO [Thread-180] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:50:36,486 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#23 average throughput is 57.59 MB/second, slept 17 time(s) and total slept time is 5562 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-16 23:50:36,572 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:50:36,600 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 1af5f51465d64fa18ccd35aeb058b519(size=69.0 M), total size for store is 48.3 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-16 23:50:36,601 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605570620700; duration=15sec 2020-11-16 23:50:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 86489 ns. 2020-11-16 23:50:46,207 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570573506 with entries=26776, filesize=128.10 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570646169 2020-11-16 23:51:15,517 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.94 MB heapSize=128.10 MB 2020-11-16 23:51:15,553 INFO [Thread-184] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:51:16,321 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.94 MB at sequenceid=270841140 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/307879e6748d4b0cb6b78fbad7207ffa 2020-11-16 23:51:16,328 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:51:16,348 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/307879e6748d4b0cb6b78fbad7207ffa, entries=318241, sequenceid=270841140, filesize=8.5 M 2020-11-16 23:51:16,354 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.94 MB/110033393, heapSize ~128.10 MB/134321160, currentSize=30.56 KB/31291 for 0469ce862509756add4a8cdb00bc465c in 837ms, sequenceid=270841140, compaction requested=true 2020-11-16 23:51:16,355 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-16 23:51:16,355 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/adb064193a994aecbe550670ea4e3d78, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/1bbdaa9e556e479cb2505a1a6077c9ca, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/307879e6748d4b0cb6b78fbad7207ffa] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=22.7 M 2020-11-16 23:51:16,390 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:51:16,411 INFO [Thread-186] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:51:16,442 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:51:20,997 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#25 average throughput is 60.48 MB/second, slept 5 time(s) and total slept time is 2154 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-16 23:51:21,086 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 20dac7bf457f40bd98c70dd4ead1de7c(size=23.3 M), total size for store is 42.5 G. This selection was in queue for 0sec, and took 4sec to execute. 2020-11-16 23:51:21,086 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605570676354; duration=4sec 2020-11-16 23:51:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 82303 ns. 2020-11-16 23:52:08,765 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:52:13,311 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:52:31,721 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570646169 with entries=39240, filesize=128.27 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570751682 2020-11-16 23:53:11,295 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=89.70 MB heapSize=128.00 MB 2020-11-16 23:53:11,342 INFO [Thread-193] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:53:12,505 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=89.70 MB at sequenceid=286257349 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/6818658f0cb24d2fa3ca09d44c46c373 2020-11-16 23:53:12,515 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:53:12,553 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/6818658f0cb24d2fa3ca09d44c46c373, entries=524079, sequenceid=286257349, filesize=8.5 M 2020-11-16 23:53:12,558 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~89.70 MB/94057488, heapSize ~128.00 MB/134220680, currentSize=105.52 KB/108051 for 5e5f05205d7e40b0141cd977dd009ba1 in 1263ms, sequenceid=286257349, compaction requested=true 2020-11-16 23:53:12,559 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-16 23:53:12,560 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/a35b54de1565415d8433e92fcf59eedf, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/db2030c57ffa40109abb7268c4e5017a, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/6818658f0cb24d2fa3ca09d44c46c373] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=41.6 M 2020-11-16 23:53:12,602 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:53:12,605 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:53:12,655 INFO [Thread-195] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:53:18,052 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:53:19,909 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#27 average throughput is 57.28 MB/second, slept 7 time(s) and total slept time is 1538 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-16 23:53:19,978 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into 46579df2b8894bcd961d52236335fdca(size=41.9 M), total size for store is 24.2 G. This selection was in queue for 0sec, and took 7sec to execute. 2020-11-16 23:53:19,978 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=10, startTime=1605570792559; duration=7sec 2020-11-16 23:53:38,745 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 328851 ns. 2020-11-16 23:53:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 73272 ns. 2020-11-16 23:53:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 78229 ns. 2020-11-16 23:53:55,520 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.42 MB heapSize=128.00 MB 2020-11-16 23:53:55,557 INFO [Thread-199] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:53:56,419 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.42 MB at sequenceid=262014866 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/652cd544a8844aff8b40205fdfaa49d3 2020-11-16 23:53:56,426 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:53:56,449 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/652cd544a8844aff8b40205fdfaa49d3, entries=354601, sequenceid=262014866, filesize=8.1 M 2020-11-16 23:53:56,451 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.42 MB/107398325, heapSize ~128.00 MB/134218656, currentSize=25.14 KB/25747 for e5dc5c620be19b02666a19c280ff582e in 931ms, sequenceid=262014866, compaction requested=true 2020-11-16 23:54:05,240 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570751682 with entries=34008, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570845216 2020-11-16 23:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.51 GB, freeSize=335.52 MB, max=1.83 GB, blockCount=39421, accesses=1924227, hits=1801782, hitRatio=93.64%, , cachingAccesses=1846586, cachingHits=1801685, cachingHitsRatio=97.57%, evictions=149, evicted=0, evictedPerRun=0.0 2020-11-16 23:54:48,919 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 50924 ns. 2020-11-16 23:54:48,919 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 26330 ns. 2020-11-16 23:56:16,099 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570845216 with entries=49039, filesize=128.33 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570976070 2020-11-16 23:56:17,365 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:56:38,844 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 89568 ns. 2020-11-16 23:56:38,844 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 70475 ns. 2020-11-16 23:56:47,957 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:56:59,374 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:57:15,849 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:57:25,212 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:57:36,280 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:57:55,806 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:58:23,274 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.96 MB heapSize=128.00 MB 2020-11-16 23:58:23,325 INFO [Thread-211] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:58:24,001 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.96 MB at sequenceid=272258671 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/d1f6870dec9f4b04bfa0ae12f0a93984 2020-11-16 23:58:24,009 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:58:24,030 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/d1f6870dec9f4b04bfa0ae12f0a93984, entries=274857, sequenceid=272258671, filesize=8.5 M 2020-11-16 23:58:24,036 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~107.96 MB/113206721, heapSize ~128.00 MB/134219736, currentSize=16.86 KB/17269 for fc93f6dcc564341dcc983bf30971f73d in 762ms, sequenceid=272258671, compaction requested=true 2020-11-16 23:58:24,038 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-16 23:58:24,038 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/c7cdef393e784ab0840c1692c158478e, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/e317723427fb41658b89ea2705fe6931, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/d1f6870dec9f4b04bfa0ae12f0a93984] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=24.0 M 2020-11-16 23:58:24,075 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:58:24,081 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:58:24,156 INFO [Thread-213] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:58:29,446 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#30 average throughput is 58.81 MB/second, slept 6 time(s) and total slept time is 2748 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-16 23:58:29,514 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into abb8378b628b42ef8010bb13284c3380(size=24.6 M), total size for store is 33.2 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-16 23:58:29,514 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605571104037; duration=5sec 2020-11-16 23:58:30,193 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.16 MB heapSize=128.00 MB 2020-11-16 23:58:30,218 INFO [Thread-215] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:58:31,084 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.16 MB at sequenceid=262028534 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/f66a077a5f694249997ea13d1f32090c 2020-11-16 23:58:31,111 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f66a077a5f694249997ea13d1f32090c, entries=401805, sequenceid=262028534, filesize=8.1 M 2020-11-16 23:58:31,114 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.16 MB/103977571, heapSize ~128.00 MB/134218296, currentSize=157.33 KB/161106 for e5dc5c620be19b02666a19c280ff582e in 920ms, sequenceid=262028534, compaction requested=true 2020-11-16 23:58:31,114 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-16 23:58:31,114 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1af5f51465d64fa18ccd35aeb058b519, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/652cd544a8844aff8b40205fdfaa49d3, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f66a077a5f694249997ea13d1f32090c] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=85.3 M 2020-11-16 23:58:31,146 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:58:31,174 INFO [Thread-217] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:58:32,813 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:58:40,679 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570976070 with entries=51706, filesize=128.21 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571120630 2020-11-16 23:58:49,597 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#32 average throughput is 59.98 MB/second, slept 22 time(s) and total slept time is 7355 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-16 23:58:49,681 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into b1ebd587292140ed80ba1b3dbbf20f55(size=87.3 M), total size for store is 48.3 G. This selection was in queue for 0sec, and took 18sec to execute. 2020-11-16 23:58:49,681 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605571111114; duration=18sec 2020-11-16 23:58:58,886 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:59:14,834 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:59:25,185 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-16 23:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=312.89 MB, max=1.83 GB, blockCount=39896, accesses=1959886, hits=1813986, hitRatio=92.56%, , cachingAccesses=1859258, cachingHits=1813882, cachingHitsRatio=97.56%, evictions=179, evicted=0, evictedPerRun=0.0 2020-11-16 23:59:38,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 355498 ns. 2020-11-16 23:59:38,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 71939 ns. 2020-11-16 23:59:38,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 55631 ns. 2020-11-16 23:59:58,733 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 73023 ns. 2020-11-16 23:59:58,733 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 54660 ns. 2020-11-17 00:00:24,784 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.39 MB heapSize=128.00 MB 2020-11-17 00:00:24,844 INFO [Thread-223] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:00:25,798 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.39 MB at sequenceid=260513312 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/3790573bc64b41c384b8bf58fab8a16c 2020-11-17 00:00:25,810 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:00:25,831 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/3790573bc64b41c384b8bf58fab8a16c, entries=568608, sequenceid=260513312, filesize=8.7 M 2020-11-17 00:00:25,833 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.39 MB/89535882, heapSize ~128.00 MB/134218968, currentSize=31.26 KB/32014 for 73aae5715542e1a5370f7540d2117abc in 1049ms, sequenceid=260513312, compaction requested=true 2020-11-17 00:00:25,834 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 00:00:25,834 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/d0bbb11af36c43cab73875bf442c8632, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/9d68b5a261d14c478646372105eeaf5a, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/3790573bc64b41c384b8bf58fab8a16c] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=88.8 M 2020-11-17 00:00:25,867 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:00:25,878 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:00:25,907 INFO [Thread-225] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:00:28,363 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571120630 with entries=40572, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571228332 2020-11-17 00:00:38,036 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:00:41,175 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#34 average throughput is 57.46 MB/second, slept 15 time(s) and total slept time is 2450 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:00:41,246 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:00:41,267 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into b55f9f42cca9455683d4a4e0a70dfe5b(size=89.5 M), total size for store is 12.3 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 00:00:41,267 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605571225833; duration=15sec 2020-11-17 00:00:50,154 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:01:39,307 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 94717 ns. 2020-11-17 00:01:39,308 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 47357 ns. 2020-11-17 00:01:55,943 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.05 MB heapSize=128.09 MB 2020-11-17 00:01:56,008 INFO [Thread-232] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:01:56,779 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.05 MB at sequenceid=262039127 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/b5e31b43419e418ab94a25627447198c 2020-11-17 00:01:56,791 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:01:56,811 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b5e31b43419e418ab94a25627447198c, entries=389837, sequenceid=262039127, filesize=8.1 M 2020-11-17 00:01:56,814 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.05 MB/104912200, heapSize ~128.09 MB/134313384, currentSize=774.70 KB/793291 for e5dc5c620be19b02666a19c280ff582e in 871ms, sequenceid=262039127, compaction requested=true 2020-11-17 00:02:01,495 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:02:05,351 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.70 MB heapSize=128.00 MB 2020-11-17 00:02:05,385 INFO [Thread-234] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:02:06,005 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:02:06,505 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.70 MB at sequenceid=290201166 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/5180f0df3ecd467bb9bac6b5e840bbea 2020-11-17 00:02:06,533 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/5180f0df3ecd467bb9bac6b5e840bbea, entries=537494, sequenceid=290201166, filesize=8.4 M 2020-11-17 00:02:06,536 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~88.70 MB/93011068, heapSize ~128.00 MB/134218184, currentSize=39.99 KB/40948 for bdaa31a63b7f3d06d9909b7560333e79 in 1184ms, sequenceid=290201166, compaction requested=true 2020-11-17 00:02:25,802 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571228332 with entries=43919, filesize=128.19 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571345767 2020-11-17 00:02:25,803 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569383863 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605569383863 2020-11-17 00:02:25,815 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569776761 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605569776761 2020-11-17 00:02:25,821 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569792885 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605569792885 2020-11-17 00:02:25,827 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569819634 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605569819634 2020-11-17 00:02:25,832 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569895159 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605569895159 2020-11-17 00:02:25,838 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605569952586 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605569952586 2020-11-17 00:02:25,844 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570041657 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605570041657 2020-11-17 00:02:25,850 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570152965 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605570152965 2020-11-17 00:02:25,856 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570257439 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605570257439 2020-11-17 00:02:25,862 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570379725 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605570379725 2020-11-17 00:02:35,201 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:03:21,412 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:03:40,565 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:04:02,316 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=315.97 MB, max=1.83 GB, blockCount=39788, accesses=1983588, hits=1823271, hitRatio=91.92%, , cachingAccesses=1868450, cachingHits=1823053, cachingHitsRatio=97.57%, evictions=209, evicted=0, evictedPerRun=0.0 2020-11-17 00:04:40,369 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571345767 with entries=49477, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571480335 2020-11-17 00:04:40,957 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.92 MB heapSize=128.06 MB 2020-11-17 00:04:40,989 INFO [Thread-242] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:04:41,894 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.92 MB at sequenceid=270880254 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/1057a1a80cc0466db9379825a25220c6 2020-11-17 00:04:41,903 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:04:41,931 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/1057a1a80cc0466db9379825a25220c6, entries=318048, sequenceid=270880254, filesize=8.6 M 2020-11-17 00:04:41,933 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.92 MB/110015808, heapSize ~128.06 MB/134279240, currentSize=45.49 KB/46586 for 0469ce862509756add4a8cdb00bc465c in 976ms, sequenceid=270880254, compaction requested=true 2020-11-17 00:04:59,391 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 19350 ns. 2020-11-17 00:04:59,391 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 51410 ns. 2020-11-17 00:04:59,919 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:08,399 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:22,586 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=104.21 MB heapSize=128.26 MB 2020-11-17 00:05:22,665 INFO [Thread-246] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:23,095 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:23,456 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.21 MB at sequenceid=262049590 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/3e104f839ab34b99be41e17fa65976a2 2020-11-17 00:05:23,485 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3e104f839ab34b99be41e17fa65976a2, entries=332991, sequenceid=262049590, filesize=8.1 M 2020-11-17 00:05:23,487 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.21 MB/109277283, heapSize ~128.26 MB/134487600, currentSize=31.35 KB/32105 for e5dc5c620be19b02666a19c280ff582e in 901ms, sequenceid=262049590, compaction requested=true 2020-11-17 00:05:23,488 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 00:05:23,488 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b1ebd587292140ed80ba1b3dbbf20f55, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b5e31b43419e418ab94a25627447198c, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3e104f839ab34b99be41e17fa65976a2] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=103.5 M 2020-11-17 00:05:23,533 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:23,536 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:23,569 INFO [Thread-248] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:24,501 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:29,241 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:30,267 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-17 00:05:30,268 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-17 00:05:30,270 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-17 00:05:36,794 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=104.45 MB heapSize=128.09 MB 2020-11-17 00:05:36,839 INFO [Thread-251] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:36,897 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:37,782 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.45 MB at sequenceid=251303383 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/c74f818794424a95a2af106de6d09a1a 2020-11-17 00:05:37,820 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/c74f818794424a95a2af106de6d09a1a, entries=336301, sequenceid=251303383, filesize=6.9 M 2020-11-17 00:05:37,829 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.45 MB/109524486, heapSize ~128.09 MB/134313088, currentSize=170.28 KB/174363 for 82421ce8c5c8b48c573baee83caa8305 in 1035ms, sequenceid=251303383, compaction requested=true 2020-11-17 00:05:37,830 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 00:05:37,831 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0576e39e8a0a45b6ae90175d859bf72b, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d1cce499794a401f803246348e2501bd, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/c74f818794424a95a2af106de6d09a1a] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=42.9 M 2020-11-17 00:05:37,902 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:37,909 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:37,949 INFO [Thread-253] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 54133 ns. 2020-11-17 00:05:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 358556 ns. 2020-11-17 00:05:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 66522 ns. 2020-11-17 00:05:54,414 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=98.29 MB heapSize=128.02 MB 2020-11-17 00:05:54,567 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#39 average throughput is 42.24 MB/second, slept 25 time(s) and total slept time is 14634 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 00:05:54,586 INFO [Thread-256] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:54,704 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 2172fe8afce541bd8e6686a68a500332(size=105.2 M), total size for store is 48.3 G. This selection was in queue for 0sec, and took 31sec to execute. 2020-11-17 00:05:54,704 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605571523487; duration=31sec 2020-11-17 00:05:55,646 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.29 MB at sequenceid=258683713 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/6ffe0fd08f8e45ec9cbf5bf4387cb812 2020-11-17 00:05:55,676 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/6ffe0fd08f8e45ec9cbf5bf4387cb812, entries=403459, sequenceid=258683713, filesize=7.7 M 2020-11-17 00:05:55,677 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.29 MB/103065907, heapSize ~128.02 MB/134235864, currentSize=287.92 KB/294833 for 06f7ca9534710e37d90d9f8345864259 in 1263ms, sequenceid=258683713, compaction requested=true 2020-11-17 00:05:55,678 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 00:05:55,678 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/cdd55b36219c4416a35b27abf997ac83, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/0a9f1278b4ae406db3a593940b2883cd, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/6ffe0fd08f8e45ec9cbf5bf4387cb812] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=48.7 M 2020-11-17 00:05:55,713 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:55,718 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:55,761 INFO [Thread-258] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:05:57,882 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#41 average throughput is 33.24 MB/second, slept 12 time(s) and total slept time is 10168 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 00:05:57,998 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 8bce4f5f01dd4e25b9f2e6e0416778ac(size=42.9 M), total size for store is 10.8 G. This selection was in queue for 0sec, and took 20sec to execute. 2020-11-17 00:05:57,998 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605571537830; duration=20sec 2020-11-17 00:06:06,653 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#43 average throughput is 55.87 MB/second, slept 11 time(s) and total slept time is 3694 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 00:06:06,759 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 3d9566df1c834853bcd0388e9a16b347(size=49.1 M), total size for store is 55.2 G. This selection was in queue for 0sec, and took 11sec to execute. 2020-11-17 00:06:06,759 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605571555678; duration=11sec 2020-11-17 00:06:19,668 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571480335 with entries=36134, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571579615 2020-11-17 00:06:19,669 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570457977 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605570457977 2020-11-17 00:06:19,677 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570573506 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605570573506 2020-11-17 00:06:19,686 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570646169 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605570646169 2020-11-17 00:06:47,958 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:07:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 76152 ns. 2020-11-17 00:07:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 100013 ns. 2020-11-17 00:07:40,181 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:07:44,495 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:07:50,361 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571579615 with entries=33470, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571670203 2020-11-17 00:08:42,696 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:08:54,105 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:09:07,887 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571670203 with entries=28384, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571747839 2020-11-17 00:09:09,016 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:09:30,308 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:09:34,663 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=314.47 MB, max=1.83 GB, blockCount=39820, accesses=2403100, hits=2200735, hitRatio=91.58%, , cachingAccesses=2245973, cachingHits=2200481, cachingHitsRatio=97.97%, evictions=239, evicted=0, evictedPerRun=0.0 2020-11-17 00:09:42,609 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:09:48,994 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:09:59,501 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18516 ns. 2020-11-17 00:09:59,502 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 48897 ns. 2020-11-17 00:10:00,615 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:10:04,204 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:10:09,029 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:10:17,427 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.69 MB heapSize=128.06 MB 2020-11-17 00:10:17,482 INFO [Thread-269] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:10:18,123 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.69 MB at sequenceid=251311643 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/92937586d4384eb58e3652a53ede3663 2020-11-17 00:10:18,135 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:10:18,164 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/92937586d4384eb58e3652a53ede3663, entries=305891, sequenceid=251311643, filesize=6.4 M 2020-11-17 00:10:18,166 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.69 MB/111872332, heapSize ~128.06 MB/134281488, currentSize=378.52 KB/387609 for 82421ce8c5c8b48c573baee83caa8305 in 740ms, sequenceid=251311643, compaction requested=true 2020-11-17 00:10:22,731 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.10 MB heapSize=128.54 MB 2020-11-17 00:10:22,770 INFO [Thread-271] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:10:23,668 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.10 MB at sequenceid=262064508 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/080d803b805e4fc9b44ad2e46c9f7492 2020-11-17 00:10:23,711 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/080d803b805e4fc9b44ad2e46c9f7492, entries=395506, sequenceid=262064508, filesize=8.2 M 2020-11-17 00:10:23,713 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.10 MB/104962764, heapSize ~128.54 MB/134785112, currentSize=850.48 KB/870889 for e5dc5c620be19b02666a19c280ff582e in 982ms, sequenceid=262064508, compaction requested=true 2020-11-17 00:10:31,634 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:10:45,256 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571747839 with entries=35727, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571845224 2020-11-17 00:10:59,551 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:11:15,512 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:11:24,846 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:11:39,194 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:11:39,412 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 71946 ns. 2020-11-17 00:11:39,412 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 214048 ns. 2020-11-17 00:11:39,412 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 80197 ns. 2020-11-17 00:11:44,624 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:12:03,555 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:12:04,513 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571845224 with entries=29152, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571924480 2020-11-17 00:12:17,449 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:12:39,120 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 68403 ns. 2020-11-17 00:12:39,120 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 64080 ns. 2020-11-17 00:13:17,739 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:13:25,841 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:13:39,967 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:13:49,036 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571924480 with entries=38674, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572028994 2020-11-17 00:13:54,862 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.10 MB heapSize=128.34 MB 2020-11-17 00:13:54,900 INFO [Thread-281] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:13:55,937 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.10 MB at sequenceid=262075164 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/7b5cef21ae4b4b54b5f9597cf2b56333 2020-11-17 00:13:55,943 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:13:55,972 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7b5cef21ae4b4b54b5f9597cf2b56333, entries=406547, sequenceid=262075164, filesize=8.1 M 2020-11-17 00:13:55,974 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~99.10 MB/103909633, heapSize ~128.34 MB/134570352, currentSize=610.38 KB/625027 for e5dc5c620be19b02666a19c280ff582e in 1112ms, sequenceid=262075164, compaction requested=true 2020-11-17 00:13:55,975 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 00:13:55,975 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/76ca3edd87814b3cb1860d063abafd8e, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/570204c9eb274a998bfc51e0b6b7efd5, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2172fe8afce541bd8e6686a68a500332, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/080d803b805e4fc9b44ad2e46c9f7492, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7b5cef21ae4b4b54b5f9597cf2b56333] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=761.7 M 2020-11-17 00:13:56,074 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:13:56,080 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:13:56,092 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:13:56,100 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:13:56,114 INFO [Thread-283] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:14:00,331 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:14:04,607 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.01 MB heapSize=128.17 MB 2020-11-17 00:14:04,641 INFO [Thread-285] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:14:05,410 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:14:05,469 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.01 MB at sequenceid=272287066 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/e62f83738bc043cabf18b3b72d2a9836 2020-11-17 00:14:05,499 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/e62f83738bc043cabf18b3b72d2a9836, entries=276539, sequenceid=272287066, filesize=8.5 M 2020-11-17 00:14:05,502 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.01 MB/113260044, heapSize ~128.17 MB/134398560, currentSize=678.02 KB/694294 for fc93f6dcc564341dcc983bf30971f73d in 895ms, sequenceid=272287066, compaction requested=true 2020-11-17 00:14:10,132 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:14:20,898 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fdaad44cfa5c4a12842dcbf2a571c2b0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=308.71 MB, max=1.83 GB, blockCount=40029, accesses=3051542, hits=2811453, hitRatio=92.13%, , cachingAccesses=2856862, cachingHits=2811147, cachingHitsRatio=98.40%, evictions=269, evicted=0, evictedPerRun=0.0 2020-11-17 00:14:41,759 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.20 MB heapSize=128.12 MB 2020-11-17 00:14:41,825 INFO [Thread-289] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:14:42,999 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.20 MB at sequenceid=260557648 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/582638f03fd54c19a5d5c5e0a486e40a 2020-11-17 00:14:43,006 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:14:43,029 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/582638f03fd54c19a5d5c5e0a486e40a, entries=574590, sequenceid=260557648, filesize=8.7 M 2020-11-17 00:14:43,031 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.20 MB/89335762, heapSize ~128.12 MB/134346976, currentSize=271.09 KB/277598 for 73aae5715542e1a5370f7540d2117abc in 1272ms, sequenceid=260557648, compaction requested=true 2020-11-17 00:14:46,800 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fdaad44cfa5c4a12842dcbf2a571c2b0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:14:55,136 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572028994 with entries=24180, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572095113 2020-11-17 00:14:56,054 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:14:59,111 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:15:08,716 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18430 ns. 2020-11-17 00:15:08,717 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 40091 ns. 2020-11-17 00:15:09,058 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fdaad44cfa5c4a12842dcbf2a571c2b0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:15:09,175 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:15:12,312 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:15:36,865 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fdaad44cfa5c4a12842dcbf2a571c2b0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:15:58,899 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.42 MB heapSize=128.21 MB 2020-11-17 00:15:58,943 INFO [Thread-297] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:15:59,528 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.42 MB at sequenceid=251321850 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/4c0a3aed9c664376a6731cc3b7e5517a 2020-11-17 00:15:59,543 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:15:59,562 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/4c0a3aed9c664376a6731cc3b7e5517a, entries=311514, sequenceid=251321850, filesize=6.4 M 2020-11-17 00:15:59,564 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.42 MB/111591437, heapSize ~128.21 MB/134435624, currentSize=460.65 KB/471709 for 82421ce8c5c8b48c573baee83caa8305 in 665ms, sequenceid=251321850, compaction requested=true 2020-11-17 00:15:59,565 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 00:15:59,565 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/8bce4f5f01dd4e25b9f2e6e0416778ac, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/92937586d4384eb58e3652a53ede3663, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/4c0a3aed9c664376a6731cc3b7e5517a] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=55.7 M 2020-11-17 00:15:59,600 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:15:59,604 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:15:59,637 INFO [Thread-299] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:16:08,057 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fdaad44cfa5c4a12842dcbf2a571c2b0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:16:08,207 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:16:08,209 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:16:08,724 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:16:13,466 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:16:13,468 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:16:16,836 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:16:27,703 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#51 average throughput is 31.21 MB/second, slept 17 time(s) and total slept time is 20038 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 00:16:27,821 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 3056afdd3fca4bf58153bdef0b3dae58(size=55.6 M), total size for store is 10.9 G. This selection was in queue for 0sec, and took 28sec to execute. 2020-11-17 00:16:27,821 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605572159565; duration=28sec 2020-11-17 00:16:39,261 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:16:39,338 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:16:41,147 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#47 average throughput is 53.41 MB/second, slept 161 time(s) and total slept time is 75402 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 00:16:41,200 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:16:41,225 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 5 file(s) in e of e5dc5c620be19b02666a19c280ff582e into fdaad44cfa5c4a12842dcbf2a571c2b0(size=763.1 M), total size for store is 48.3 G. This selection was in queue for 0sec, and took 2mins, 45sec to execute. 2020-11-17 00:16:41,225 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605572035974; duration=2mins, 45sec 2020-11-17 00:16:46,159 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:16:47,529 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.78 MB heapSize=128.00 MB 2020-11-17 00:16:47,566 INFO [Thread-305] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:16:47,798 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572095113 with entries=43344, filesize=128.10 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572207776 2020-11-17 00:16:48,570 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.78 MB at sequenceid=286335177 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/75fe0b9b2f674dfd9eb78ed8558a2928 2020-11-17 00:16:48,604 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/75fe0b9b2f674dfd9eb78ed8558a2928, entries=519643, sequenceid=286335177, filesize=9.0 M 2020-11-17 00:16:48,605 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.78 MB/95192692, heapSize ~128.00 MB/134218272, currentSize=47.98 KB/49129 for 5e5f05205d7e40b0141cd977dd009ba1 in 1077ms, sequenceid=286335177, compaction requested=true 2020-11-17 00:17:01,302 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.83 MB heapSize=128.04 MB 2020-11-17 00:17:01,351 INFO [Thread-308] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:17:02,161 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.83 MB at sequenceid=258718638 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/9897403432a141ed9501367c60fd55a9 2020-11-17 00:17:02,169 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:17:02,204 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/9897403432a141ed9501367c60fd55a9, entries=372934, sequenceid=258718638, filesize=7.1 M 2020-11-17 00:17:02,206 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.83 MB/105729705, heapSize ~128.04 MB/134261616, currentSize=425.83 KB/436053 for 06f7ca9534710e37d90d9f8345864259 in 903ms, sequenceid=258718638, compaction requested=true 2020-11-17 00:17:15,122 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.98 MB heapSize=128.00 MB 2020-11-17 00:17:15,160 INFO [Thread-311] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:17:15,937 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.98 MB at sequenceid=262085609 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/85e0c3aa7e164378be04eea10a70e482 2020-11-17 00:17:15,944 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:17:15,972 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/85e0c3aa7e164378be04eea10a70e482, entries=360759, sequenceid=262085609, filesize=8.2 M 2020-11-17 00:17:15,974 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.98 MB/106930125, heapSize ~128.00 MB/134220648, currentSize=570.29 KB/583975 for e5dc5c620be19b02666a19c280ff582e in 852ms, sequenceid=262085609, compaction requested=true 2020-11-17 00:17:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 210580 ns. 2020-11-17 00:17:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 87792 ns. 2020-11-17 00:17:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 74486 ns. 2020-11-17 00:17:44,375 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:17:54,932 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:17:58,709 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:18:04,756 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572207776 with entries=28590, filesize=128.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572284718 2020-11-17 00:18:04,757 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570751682 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605570751682 2020-11-17 00:18:04,772 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570845216 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605570845216 2020-11-17 00:18:04,779 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605570976070 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605570976070 2020-11-17 00:18:04,786 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571120630 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605571120630 2020-11-17 00:18:12,559 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:18:18,199 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.03 MB heapSize=128.35 MB 2020-11-17 00:18:18,243 INFO [Thread-315] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:18:19,052 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.03 MB at sequenceid=270919803 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/914e9b40e0b24f91b6ed14c1ed27d426 2020-11-17 00:18:19,059 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:18:19,080 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/914e9b40e0b24f91b6ed14c1ed27d426, entries=320547, sequenceid=270919803, filesize=8.7 M 2020-11-17 00:18:19,082 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.03 MB/110130724, heapSize ~128.35 MB/134581368, currentSize=26.10 KB/26722 for 0469ce862509756add4a8cdb00bc465c in 883ms, sequenceid=270919803, compaction requested=true 2020-11-17 00:18:19,083 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 00:18:19,083 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/20dac7bf457f40bd98c70dd4ead1de7c, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/1057a1a80cc0466db9379825a25220c6, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/914e9b40e0b24f91b6ed14c1ed27d426] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=40.6 M 2020-11-17 00:18:19,117 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:18:19,125 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:18:19,146 INFO [Thread-317] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:18:27,317 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#56 average throughput is 59.55 MB/second, slept 9 time(s) and total slept time is 3569 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:18:27,398 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 61b273e627884b1c9238833b4feb3ad6(size=41.4 M), total size for store is 42.5 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 00:18:27,399 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605572299082; duration=8sec 2020-11-17 00:18:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 69576 ns. 2020-11-17 00:18:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 86557 ns. 2020-11-17 00:18:55,355 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:19:09,011 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=316.14 MB, max=1.83 GB, blockCount=39784, accesses=3185190, hits=2817652, hitRatio=88.46%, , cachingAccesses=2862869, cachingHits=2817115, cachingHitsRatio=98.40%, evictions=299, evicted=0, evictedPerRun=0.0 2020-11-17 00:19:53,092 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572284718 with entries=40579, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572393039 2020-11-17 00:20:09,109 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15670 ns. 2020-11-17 00:20:09,109 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 46510 ns. 2020-11-17 00:20:45,572 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:20:49,870 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.02 MB heapSize=128.07 MB 2020-11-17 00:20:49,923 INFO [Thread-325] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:20:50,976 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.02 MB at sequenceid=262096399 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/de23d4afbad74a8dbf30af0482c12925 2020-11-17 00:20:51,010 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/de23d4afbad74a8dbf30af0482c12925, entries=361355, sequenceid=262096399, filesize=8.2 M 2020-11-17 00:20:51,019 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.02 MB/106973747, heapSize ~128.07 MB/134291680, currentSize=855.06 KB/875585 for e5dc5c620be19b02666a19c280ff582e in 1149ms, sequenceid=262096399, compaction requested=true 2020-11-17 00:21:06,870 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572393039 with entries=27195, filesize=128.29 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572466819 2020-11-17 00:21:38,267 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.28 MB heapSize=128.16 MB 2020-11-17 00:21:38,303 INFO [Thread-329] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:21:39,016 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.28 MB at sequenceid=251331989 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/d6a8ce21d6c5484ebb5a10cf29dd0472 2020-11-17 00:21:39,022 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:21:39,045 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d6a8ce21d6c5484ebb5a10cf29dd0472, entries=312735, sequenceid=251331989, filesize=6.4 M 2020-11-17 00:21:39,047 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.28 MB/111442995, heapSize ~128.16 MB/134388672, currentSize=40.41 KB/41379 for 82421ce8c5c8b48c573baee83caa8305 in 781ms, sequenceid=251331989, compaction requested=true 2020-11-17 00:22:37,636 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:23:03,796 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572466819 with entries=43301, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572583745 2020-11-17 00:23:13,762 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:23:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 72999 ns. 2020-11-17 00:23:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 92451 ns. 2020-11-17 00:23:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 288250 ns. 2020-11-17 00:23:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 80990 ns. 2020-11-17 00:23:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 89470 ns. 2020-11-17 00:23:47,229 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:23:56,099 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:24:19,303 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572583745 with entries=27964, filesize=128.53 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572659254 2020-11-17 00:24:26,955 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:24:34,412 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:24:36,025 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=315.85 MB, max=1.83 GB, blockCount=39787, accesses=3455017, hits=3087453, hitRatio=89.36%, , cachingAccesses=3132696, cachingHits=3086916, cachingHitsRatio=98.54%, evictions=329, evicted=0, evictedPerRun=0.0 2020-11-17 00:25:17,717 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.94 MB heapSize=128.05 MB 2020-11-17 00:25:17,753 INFO [Thread-339] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:25:18,544 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.94 MB at sequenceid=262109905 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/68b16b4c369a4c72ac6909f0e9df53f3 2020-11-17 00:25:18,551 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:25:18,576 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/68b16b4c369a4c72ac6909f0e9df53f3, entries=376564, sequenceid=262109905, filesize=8.1 M 2020-11-17 00:25:18,578 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.94 MB/105842034, heapSize ~128.05 MB/134272416, currentSize=225.42 KB/230830 for e5dc5c620be19b02666a19c280ff582e in 861ms, sequenceid=262109905, compaction requested=true 2020-11-17 00:25:18,579 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 00:25:18,579 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/85e0c3aa7e164378be04eea10a70e482, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/de23d4afbad74a8dbf30af0482c12925, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/68b16b4c369a4c72ac6909f0e9df53f3] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.5 M 2020-11-17 00:25:18,614 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:25:18,618 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:25:18,653 INFO [Thread-341] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:25:18,723 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 22952 ns. 2020-11-17 00:25:18,723 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 45653 ns. 2020-11-17 00:25:23,886 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#60 average throughput is 57.99 MB/second, slept 6 time(s) and total slept time is 1945 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:25:23,970 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 85d70be225b1424c8048e61016b8e4be(size=25.1 M), total size for store is 48.4 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 00:25:23,970 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605572718578; duration=5sec 2020-11-17 00:26:06,982 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572659254 with entries=39104, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572766916 2020-11-17 00:26:20,318 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:27:26,393 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572766916 with entries=29972, filesize=128.33 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572846345 2020-11-17 00:28:05,078 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:28:10,428 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.00 MB heapSize=128.02 MB 2020-11-17 00:28:10,460 INFO [Thread-351] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:28:11,140 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.00 MB at sequenceid=251343679 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/66546e05e39c424b8aad04de01da1ec0 2020-11-17 00:28:11,148 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:28:11,177 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/66546e05e39c424b8aad04de01da1ec0, entries=314570, sequenceid=251343679, filesize=6.4 M 2020-11-17 00:28:11,179 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.00 MB/111150482, heapSize ~128.02 MB/134241424, currentSize=276.00 KB/282621 for 82421ce8c5c8b48c573baee83caa8305 in 752ms, sequenceid=251343679, compaction requested=true 2020-11-17 00:28:11,180 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 00:28:11,180 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/3056afdd3fca4bf58153bdef0b3dae58, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d6a8ce21d6c5484ebb5a10cf29dd0472, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/66546e05e39c424b8aad04de01da1ec0] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=68.5 M 2020-11-17 00:28:11,222 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:28:11,226 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:28:11,259 INFO [Thread-353] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:28:30,595 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#62 average throughput is 56.24 MB/second, slept 21 time(s) and total slept time is 8665 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:28:30,647 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:28:30,665 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into bdbd77e142f6418aa48411afd0948f00(size=68.4 M), total size for store is 10.9 G. This selection was in queue for 0sec, and took 19sec to execute. 2020-11-17 00:28:30,665 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605572891179; duration=19sec 2020-11-17 00:28:39,275 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 71241 ns. 2020-11-17 00:28:39,275 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 73748 ns. 2020-11-17 00:28:45,042 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.03 MB heapSize=128.00 MB 2020-11-17 00:28:45,084 INFO [Thread-357] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:28:45,832 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.03 MB at sequenceid=258755471 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/f3a88b44d9174178b6978e3bc2d4349d 2020-11-17 00:28:45,838 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:28:45,857 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/f3a88b44d9174178b6978e3bc2d4349d, entries=379246, sequenceid=258755471, filesize=7.2 M 2020-11-17 00:28:45,858 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.03 MB/105933662, heapSize ~128.00 MB/134218064, currentSize=27.77 KB/28437 for 06f7ca9534710e37d90d9f8345864259 in 816ms, sequenceid=258755471, compaction requested=true 2020-11-17 00:28:45,859 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 00:28:45,859 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/3d9566df1c834853bcd0388e9a16b347, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/9897403432a141ed9501367c60fd55a9, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/f3a88b44d9174178b6978e3bc2d4349d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=63.4 M 2020-11-17 00:28:45,888 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:28:45,893 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:28:45,916 INFO [Thread-359] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:28:58,209 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.24 MB heapSize=128.05 MB 2020-11-17 00:28:58,261 INFO [Thread-362] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:28:59,171 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.24 MB at sequenceid=262120925 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/b62775168a7244f9b3d5de7275e7602d 2020-11-17 00:28:59,178 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:28:59,198 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b62775168a7244f9b3d5de7275e7602d, entries=414890, sequenceid=262120925, filesize=8.0 M 2020-11-17 00:28:59,200 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.24 MB/103012912, heapSize ~128.05 MB/134275304, currentSize=286.23 KB/293098 for e5dc5c620be19b02666a19c280ff582e in 991ms, sequenceid=262120925, compaction requested=true 2020-11-17 00:28:59,934 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#64 average throughput is 57.74 MB/second, slept 16 time(s) and total slept time is 5235 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:28:59,994 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 3c66244b722647a78d4dda6b2cf1ee9e(size=63.7 M), total size for store is 55.2 G. This selection was in queue for 0sec, and took 14sec to execute. 2020-11-17 00:28:59,994 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605572925859; duration=14sec 2020-11-17 00:29:03,102 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=84.61 MB heapSize=128.11 MB 2020-11-17 00:29:03,133 INFO [Thread-364] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:29:04,221 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=84.61 MB at sequenceid=260602745 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/d1a149b68c264491a53d4dc8dfafc147 2020-11-17 00:29:04,227 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:29:04,250 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/d1a149b68c264491a53d4dc8dfafc147, entries=575853, sequenceid=260602745, filesize=8.6 M 2020-11-17 00:29:04,252 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~84.61 MB/88718250, heapSize ~128.11 MB/134333288, currentSize=262.48 KB/268778 for 73aae5715542e1a5370f7540d2117abc in 1150ms, sequenceid=260602745, compaction requested=true 2020-11-17 00:29:04,253 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 00:29:04,253 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/b55f9f42cca9455683d4a4e0a70dfe5b, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/582638f03fd54c19a5d5c5e0a486e40a, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/d1a149b68c264491a53d4dc8dfafc147] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=106.8 M 2020-11-17 00:29:04,319 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:29:04,324 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:29:04,351 INFO [Thread-366] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:29:11,077 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572846345 with entries=38231, filesize=128.19 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572951057 2020-11-17 00:29:11,511 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:29:19,026 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:29:22,822 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#67 average throughput is 56.11 MB/second, slept 15 time(s) and total slept time is 2799 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:29:22,893 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:29:22,913 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 26eb2bad19164edbad2c808cc5824c8c(size=107.5 M), total size for store is 12.4 G. This selection was in queue for 0sec, and took 18sec to execute. 2020-11-17 00:29:22,913 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605572944252; duration=18sec 2020-11-17 00:29:26,258 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.22 MB heapSize=128.47 MB 2020-11-17 00:29:26,285 INFO [Thread-368] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:29:26,933 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.22 MB at sequenceid=272315248 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/9cb4a41121c941ee90789be60d1b2d0f 2020-11-17 00:29:26,939 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:29:26,961 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/9cb4a41121c941ee90789be60d1b2d0f, entries=277958, sequenceid=272315248, filesize=8.5 M 2020-11-17 00:29:26,966 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.22 MB/113477658, heapSize ~128.47 MB/134715240, currentSize=500.01 KB/512015 for fc93f6dcc564341dcc983bf30971f73d in 708ms, sequenceid=272315248, compaction requested=true 2020-11-17 00:29:26,967 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 00:29:26,967 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/abb8378b628b42ef8010bb13284c3380, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/e62f83738bc043cabf18b3b72d2a9836, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/9cb4a41121c941ee90789be60d1b2d0f] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=41.7 M 2020-11-17 00:29:27,046 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:29:27,050 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:29:27,069 INFO [Thread-370] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:29:34,388 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=315.29 MB, max=1.83 GB, blockCount=39798, accesses=3521059, hits=3092270, hitRatio=87.82%, , cachingAccesses=3137408, cachingHits=3091610, cachingHitsRatio=98.54%, evictions=359, evicted=0, evictedPerRun=0.0 2020-11-17 00:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 00:29:36,121 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#69 average throughput is 58.24 MB/second, slept 9 time(s) and total slept time is 4097 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:29:36,224 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into dc5d10802b5340d7b6f98e15d4a31971(size=42.2 M), total size for store is 33.2 G. This selection was in queue for 0sec, and took 9sec to execute. 2020-11-17 00:29:36,224 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605572966966; duration=9sec 2020-11-17 00:29:39,065 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 295133 ns. 2020-11-17 00:29:39,065 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 77222 ns. 2020-11-17 00:29:39,065 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 78070 ns. 2020-11-17 00:29:41,028 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:30:18,728 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18509 ns. 2020-11-17 00:30:18,729 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 39922 ns. 2020-11-17 00:30:26,725 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572951057 with entries=29128, filesize=128.60 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573026683 2020-11-17 00:31:50,955 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.01 MB heapSize=128.00 MB 2020-11-17 00:31:50,984 INFO [Thread-378] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:31:51,756 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.01 MB at sequenceid=270959214 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/dbdbf14f363744b78dccb5b38d025dc5 2020-11-17 00:31:51,763 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:31:51,783 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/dbdbf14f363744b78dccb5b38d025dc5, entries=316027, sequenceid=270959214, filesize=8.7 M 2020-11-17 00:31:51,785 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.01 MB/110108625, heapSize ~128.00 MB/134218944, currentSize=39.32 KB/40266 for 0469ce862509756add4a8cdb00bc465c in 830ms, sequenceid=270959214, compaction requested=true 2020-11-17 00:32:07,327 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573026683 with entries=37029, filesize=128.28 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573127291 2020-11-17 00:32:09,775 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:32:23,684 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.40 MB heapSize=128.04 MB 2020-11-17 00:32:23,718 INFO [Thread-382] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:32:24,420 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.40 MB at sequenceid=262131529 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/ee46f5e4f7dd4ddc9b77bd10a47e46be 2020-11-17 00:32:24,426 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:32:24,446 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ee46f5e4f7dd4ddc9b77bd10a47e46be, entries=368718, sequenceid=262131529, filesize=8.1 M 2020-11-17 00:32:24,448 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.40 MB/106323367, heapSize ~128.04 MB/134263848, currentSize=805.86 KB/825201 for e5dc5c620be19b02666a19c280ff582e in 765ms, sequenceid=262131529, compaction requested=true 2020-11-17 00:32:24,449 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 00:32:24,450 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/85d70be225b1424c8048e61016b8e4be, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b62775168a7244f9b3d5de7275e7602d, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ee46f5e4f7dd4ddc9b77bd10a47e46be] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.2 M 2020-11-17 00:32:24,487 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:32:24,490 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:32:24,522 INFO [Thread-384] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:32:33,235 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#72 average throughput is 57.76 MB/second, slept 9 time(s) and total slept time is 2804 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:32:33,309 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 2656da0e5cee4bd790d7e32e07ca878e(size=42.6 M), total size for store is 48.4 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 00:32:33,310 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605573144449; duration=8sec 2020-11-17 00:32:41,456 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.15 MB heapSize=128.00 MB 2020-11-17 00:32:41,491 INFO [Thread-386] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:32:42,649 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.15 MB at sequenceid=290297485 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/c094b32a709e4be2a2f7d9469bd602c4 2020-11-17 00:32:42,657 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:32:42,681 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/c094b32a709e4be2a2f7d9469bd602c4, entries=552710, sequenceid=290297485, filesize=8.8 M 2020-11-17 00:32:42,682 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~88.15 MB/92432982, heapSize ~128.00 MB/134218216, currentSize=45.79 KB/46885 for bdaa31a63b7f3d06d9909b7560333e79 in 1227ms, sequenceid=290297485, compaction requested=true 2020-11-17 00:32:42,683 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-17 00:32:42,684 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/d5b40a92e68849e6ba96b1cb743ab459, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/5180f0df3ecd467bb9bac6b5e840bbea, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/c094b32a709e4be2a2f7d9469bd602c4] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp, totalSize=48.7 M 2020-11-17 00:32:42,721 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:32:42,734 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:32:42,752 INFO [Thread-388] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:32:51,517 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: bdaa31a63b7f3d06d9909b7560333e79#e#compaction#74 average throughput is 56.72 MB/second, slept 6 time(s) and total slept time is 1517 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:32:51,611 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of bdaa31a63b7f3d06d9909b7560333e79 into d5709fce1e79448d93769d4871586dcd(size=49.2 M), total size for store is 31.7 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 00:32:51,611 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., storeName=e, priority=11, startTime=1605573162683; duration=8sec 2020-11-17 00:33:18,044 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:33:23,203 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.20 MB heapSize=128.09 MB 2020-11-17 00:33:23,239 INFO [Thread-391] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:33:23,942 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.20 MB at sequenceid=251353096 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/61782517f2d74a09aaa54d3d4c7e8793 2020-11-17 00:33:23,948 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:33:23,965 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/61782517f2d74a09aaa54d3d4c7e8793, entries=312132, sequenceid=251353096, filesize=6.4 M 2020-11-17 00:33:23,967 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.20 MB/111357043, heapSize ~128.09 MB/134315552, currentSize=113.69 KB/116415 for 82421ce8c5c8b48c573baee83caa8305 in 764ms, sequenceid=251353096, compaction requested=true 2020-11-17 00:33:25,444 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573127291 with entries=29836, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573205423 2020-11-17 00:33:25,445 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571228332 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605571228332 2020-11-17 00:33:25,467 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571345767 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605571345767 2020-11-17 00:33:25,476 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571480335 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605571480335 2020-11-17 00:33:25,482 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571579615 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605571579615 2020-11-17 00:33:25,488 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571670203 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605571670203 2020-11-17 00:33:25,495 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571747839 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605571747839 2020-11-17 00:33:25,500 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571845224 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605571845224 2020-11-17 00:33:25,505 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605571924480 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605571924480 2020-11-17 00:33:25,509 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572028994 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605572028994 2020-11-17 00:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.52 GB, freeSize=322.53 MB, max=1.83 GB, blockCount=39576, accesses=3539761, hits=3094523, hitRatio=87.42%, , cachingAccesses=3139585, cachingHits=3093767, cachingHitsRatio=98.54%, evictions=389, evicted=0, evictedPerRun=0.0 2020-11-17 00:34:39,056 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 88614 ns. 2020-11-17 00:34:39,056 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 62109 ns. 2020-11-17 00:35:04,479 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573205423 with entries=36399, filesize=128.32 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573304425 2020-11-17 00:35:28,814 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 17655 ns. 2020-11-17 00:35:28,814 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 54431 ns. 2020-11-17 00:35:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 243112 ns. 2020-11-17 00:35:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 58167 ns. 2020-11-17 00:35:38,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 63086 ns. 2020-11-17 00:35:40,719 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.18 MB heapSize=128.60 MB 2020-11-17 00:35:40,773 INFO [Thread-399] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:35:41,547 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.18 MB at sequenceid=262141467 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/07b44804beed4b42b15917bc7c432ae3 2020-11-17 00:35:41,554 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:35:41,575 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/07b44804beed4b42b15917bc7c432ae3, entries=366371, sequenceid=262141467, filesize=8.2 M 2020-11-17 00:35:41,576 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.18 MB/107143891, heapSize ~128.60 MB/134843512, currentSize=994.23 KB/1018096 for e5dc5c620be19b02666a19c280ff582e in 857ms, sequenceid=262141467, compaction requested=true 2020-11-17 00:36:01,007 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:36:07,774 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:36:07,847 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573304425 with entries=23159, filesize=128.09 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573367799 2020-11-17 00:36:07,897 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:37:48,146 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573367799 with entries=37213, filesize=128.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573468108 2020-11-17 00:37:53,290 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.40 MB heapSize=128.13 MB 2020-11-17 00:37:53,318 INFO [Thread-406] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:37:53,922 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.40 MB at sequenceid=251361207 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/65c9ec3bf4a449eda0fa47a57b743f89 2020-11-17 00:37:53,929 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:37:53,952 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/65c9ec3bf4a449eda0fa47a57b743f89, entries=310668, sequenceid=251361207, filesize=6.3 M 2020-11-17 00:37:53,954 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.40 MB/111569016, heapSize ~128.13 MB/134355432, currentSize=550.40 KB/563614 for 82421ce8c5c8b48c573baee83caa8305 in 664ms, sequenceid=251361207, compaction requested=true 2020-11-17 00:37:53,955 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 00:37:53,956 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/bdbd77e142f6418aa48411afd0948f00, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/61782517f2d74a09aaa54d3d4c7e8793, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/65c9ec3bf4a449eda0fa47a57b743f89] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=81.1 M 2020-11-17 00:37:53,987 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:37:53,991 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:37:54,017 INFO [Thread-408] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:38:14,909 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:38:14,912 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:38:17,565 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#78 average throughput is 55.20 MB/second, slept 23 time(s) and total slept time is 9813 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:38:17,652 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 8c12f401c59f40d39cc4aa739e2bf3f0(size=81.0 M), total size for store is 10.9 G. This selection was in queue for 0sec, and took 23sec to execute. 2020-11-17 00:38:17,652 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605573473954; duration=23sec 2020-11-17 00:38:33,687 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.19 MB heapSize=128.00 MB 2020-11-17 00:38:33,742 INFO [Thread-412] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:38:34,736 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.19 MB at sequenceid=258786392 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/f61e4665e57046f1bb51f09a7c993b60 2020-11-17 00:38:34,752 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:38:34,778 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/f61e4665e57046f1bb51f09a7c993b60, entries=371495, sequenceid=258786392, filesize=7.0 M 2020-11-17 00:38:34,780 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.19 MB/106102012, heapSize ~128.00 MB/134217744, currentSize=358.28 KB/366876 for 06f7ca9534710e37d90d9f8345864259 in 1092ms, sequenceid=258786392, compaction requested=true 2020-11-17 00:38:44,868 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573468108 with entries=21073, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573524841 2020-11-17 00:39:31,731 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.74 MB heapSize=128.00 MB 2020-11-17 00:39:31,785 INFO [Thread-416] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:39:32,526 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.74 MB at sequenceid=262153094 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/c3cea3bde3b04145a5f7ff203fb932a0 2020-11-17 00:39:32,533 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:39:32,550 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c3cea3bde3b04145a5f7ff203fb932a0, entries=364048, sequenceid=262153094, filesize=8.1 M 2020-11-17 00:39:32,552 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.74 MB/106686496, heapSize ~128.00 MB/134221280, currentSize=49.43 KB/50613 for e5dc5c620be19b02666a19c280ff582e in 820ms, sequenceid=262153094, compaction requested=true 2020-11-17 00:39:32,552 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 00:39:32,553 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2656da0e5cee4bd790d7e32e07ca878e, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/07b44804beed4b42b15917bc7c432ae3, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c3cea3bde3b04145a5f7ff203fb932a0] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=58.8 M 2020-11-17 00:39:32,598 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:39:32,609 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:39:32,652 INFO [Thread-418] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.52 GB, freeSize=321.10 MB, max=1.83 GB, blockCount=39601, accesses=3817791, hits=3348176, hitRatio=87.70%, , cachingAccesses=3393257, cachingHits=3347414, cachingHitsRatio=98.65%, evictions=419, evicted=0, evictedPerRun=0.0 2020-11-17 00:39:44,880 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#81 average throughput is 57.83 MB/second, slept 14 time(s) and total slept time is 4167 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:39:44,956 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 7874e9fbeda34388bcbfc0487e635eb0(size=60.1 M), total size for store is 48.4 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 00:39:44,956 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605573572552; duration=12sec 2020-11-17 00:40:22,059 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573524841 with entries=36055, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573622015 2020-11-17 00:40:23,633 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:40:29,482 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 54653 ns. 2020-11-17 00:40:29,483 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14203 ns. 2020-11-17 00:40:35,306 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.07 MB heapSize=128.00 MB 2020-11-17 00:40:35,370 INFO [Thread-422] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:40:36,389 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.07 MB at sequenceid=286412972 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/468168d6cfeb4647935bccc434c6fe88 2020-11-17 00:40:36,396 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:40:36,413 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/468168d6cfeb4647935bccc434c6fe88, entries=529380, sequenceid=286412972, filesize=9.1 M 2020-11-17 00:40:36,415 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.07 MB/94441048, heapSize ~128.00 MB/134220024, currentSize=53.75 KB/55035 for 5e5f05205d7e40b0141cd977dd009ba1 in 1109ms, sequenceid=286412972, compaction requested=true 2020-11-17 00:40:36,416 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 00:40:36,416 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/46579df2b8894bcd961d52236335fdca, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/75fe0b9b2f674dfd9eb78ed8558a2928, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/468168d6cfeb4647935bccc434c6fe88] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=60.0 M 2020-11-17 00:40:36,451 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:40:36,456 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:40:36,481 INFO [Thread-424] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:40:38,725 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 91659 ns. 2020-11-17 00:40:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 82800 ns. 2020-11-17 00:40:44,637 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:40:46,583 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#83 average throughput is 58.91 MB/second, slept 8 time(s) and total slept time is 2379 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 00:40:46,654 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into 62b5d05f3a254213a184a144e56cc2d5(size=60.9 M), total size for store is 24.2 G. This selection was in queue for 0sec, and took 10sec to execute. 2020-11-17 00:40:46,654 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=10, startTime=1605573636416; duration=10sec 2020-11-17 00:41:07,109 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.51 MB heapSize=128.01 MB 2020-11-17 00:41:07,155 INFO [Thread-427] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:41:07,656 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.51 MB at sequenceid=251367119 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/a6dc5a17221f4e5aa53cdbab3625226b 2020-11-17 00:41:07,663 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:41:07,684 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a6dc5a17221f4e5aa53cdbab3625226b, entries=307544, sequenceid=251367119, filesize=6.2 M 2020-11-17 00:41:07,688 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.51 MB/111686193, heapSize ~128.01 MB/134231240, currentSize=208.02 KB/213010 for 82421ce8c5c8b48c573baee83caa8305 in 579ms, sequenceid=251367119, compaction requested=true 2020-11-17 00:41:22,066 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573622015 with entries=22606, filesize=128.64 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573682040 2020-11-17 00:41:22,067 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572095113 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605572095113 2020-11-17 00:41:22,077 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572207776 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605572207776 2020-11-17 00:41:22,082 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572284718 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605572284718 2020-11-17 00:41:22,090 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572393039 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605572393039 2020-11-17 00:41:22,097 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572466819 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605572466819 2020-11-17 00:41:22,102 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572583745 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605572583745 2020-11-17 00:41:22,108 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572659254 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605572659254 2020-11-17 00:41:22,113 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572766916 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605572766916 2020-11-17 00:41:38,745 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 202444 ns. 2020-11-17 00:41:38,745 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 53834 ns. 2020-11-17 00:41:38,754 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 61698 ns. 2020-11-17 00:42:21,058 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:42:25,159 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573682040 with entries=23449, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573745114 2020-11-17 00:42:54,345 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:43:05,670 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:43:12,969 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:43:22,761 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:43:26,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:43:31,914 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:43:39,088 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:43:44,747 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:43:44,750 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:43:55,841 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=97.50 MB heapSize=128.19 MB 2020-11-17 00:43:55,882 INFO [Thread-438] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:43:56,777 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.50 MB at sequenceid=262166462 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/6bf4e974e5e644af874bb7e8c4920173 2020-11-17 00:43:56,813 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6bf4e974e5e644af874bb7e8c4920173, entries=426903, sequenceid=262166462, filesize=7.9 M 2020-11-17 00:43:56,815 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~97.50 MB/102233582, heapSize ~128.19 MB/134419280, currentSize=306.38 KB/313735 for e5dc5c620be19b02666a19c280ff582e in 974ms, sequenceid=262166462, compaction requested=true 2020-11-17 00:43:58,354 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.57 MB heapSize=128.04 MB 2020-11-17 00:43:58,391 INFO [Thread-440] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:43:59,659 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.57 MB at sequenceid=260648998 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/c5b5d1fb0ac949c7ad1a4c08dbd84397 2020-11-17 00:43:59,685 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/c5b5d1fb0ac949c7ad1a4c08dbd84397, entries=574407, sequenceid=260648998, filesize=8.8 M 2020-11-17 00:43:59,687 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.57 MB/89728279, heapSize ~128.04 MB/134255816, currentSize=140.86 KB/144241 for 73aae5715542e1a5370f7540d2117abc in 1333ms, sequenceid=260648998, compaction requested=true 2020-11-17 00:44:09,682 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:44:20,538 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573745114 with entries=41795, filesize=128.37 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573860489 2020-11-17 00:44:20,538 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572846345 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605572846345 2020-11-17 00:44:27,990 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:44:35,507 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=314.51 MB, max=1.83 GB, blockCount=39801, accesses=4287898, hits=3799998, hitRatio=88.62%, , cachingAccesses=3845327, cachingHits=3799205, cachingHitsRatio=98.80%, evictions=449, evicted=0, evictedPerRun=0.0 2020-11-17 00:44:43,317 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:44:49,585 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.57 MB heapSize=128.00 MB 2020-11-17 00:44:49,645 INFO [Thread-444] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:44:50,849 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.57 MB at sequenceid=272343425 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/328927fa96354fc297b0108f0f8ebd3c 2020-11-17 00:44:50,859 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:44:50,890 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/328927fa96354fc297b0108f0f8ebd3c, entries=280093, sequenceid=272343425, filesize=8.4 M 2020-11-17 00:44:50,892 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~107.57 MB/112796123, heapSize ~128.00 MB/134218408, currentSize=18.69 KB/19139 for fc93f6dcc564341dcc983bf30971f73d in 1307ms, sequenceid=272343425, compaction requested=true 2020-11-17 00:44:58,819 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:45:19,001 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:45:29,661 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.35 MB heapSize=128.61 MB 2020-11-17 00:45:29,709 INFO [Thread-448] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:45:30,176 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:45:30,487 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.35 MB at sequenceid=270999001 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/544fba9a1cd0431fb9d3f9ad7cdfe844 2020-11-17 00:45:30,520 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/544fba9a1cd0431fb9d3f9ad7cdfe844, entries=318944, sequenceid=270999001, filesize=8.7 M 2020-11-17 00:45:30,522 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.35 MB/110470341, heapSize ~128.61 MB/134857216, currentSize=18.30 KB/18738 for 0469ce862509756add4a8cdb00bc465c in 861ms, sequenceid=270999001, compaction requested=true 2020-11-17 00:45:30,523 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 00:45:30,524 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/61b273e627884b1c9238833b4feb3ad6, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/dbdbf14f363744b78dccb5b38d025dc5, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/544fba9a1cd0431fb9d3f9ad7cdfe844] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=58.8 M 2020-11-17 00:45:30,558 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:45:30,564 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:45:30,575 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:45:30,587 INFO [Thread-450] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:45:39,608 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 19373 ns. 2020-11-17 00:45:39,608 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 56701 ns. 2020-11-17 00:45:39,608 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 58665 ns. 2020-11-17 00:45:39,609 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 79115 ns. 2020-11-17 00:45:40,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:45:42,598 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#89 average throughput is 57.96 MB/second, slept 12 time(s) and total slept time is 4500 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 00:45:42,702 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into e9102b0b74ce4a15a38c482b8bd2b94b(size=59.7 M), total size for store is 42.6 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 00:45:42,702 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605573930522; duration=12sec 2020-11-17 00:45:59,844 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:46:06,663 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573860489 with entries=39165, filesize=128.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573966616 2020-11-17 00:46:06,663 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605572951057 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605572951057 2020-11-17 00:46:06,675 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573026683 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605573026683 2020-11-17 00:46:20,263 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:46:33,471 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:46:42,689 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:05,409 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:10,277 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.18 MB heapSize=128.04 MB 2020-11-17 00:47:10,311 INFO [Thread-457] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:10,963 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.18 MB at sequenceid=251377934 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/932e43cb659d4311803a0db9404d854f 2020-11-17 00:47:10,969 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:10,990 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/932e43cb659d4311803a0db9404d854f, entries=312462, sequenceid=251377934, filesize=6.4 M 2020-11-17 00:47:10,992 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.18 MB/111333990, heapSize ~128.04 MB/134258224, currentSize=207.89 KB/212879 for 82421ce8c5c8b48c573baee83caa8305 in 714ms, sequenceid=251377934, compaction requested=true 2020-11-17 00:47:10,992 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 00:47:10,993 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/8c12f401c59f40d39cc4aa739e2bf3f0, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a6dc5a17221f4e5aa53cdbab3625226b, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/932e43cb659d4311803a0db9404d854f] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=93.6 M 2020-11-17 00:47:11,029 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:11,032 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:11,058 INFO [Thread-459] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:25,777 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:29,454 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.76 MB heapSize=128.08 MB 2020-11-17 00:47:29,527 INFO [Thread-462] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:30,300 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.76 MB at sequenceid=262177331 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/2217ecd985df43b7922d5bd83fdbb1b7 2020-11-17 00:47:30,307 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:30,328 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2217ecd985df43b7922d5bd83fdbb1b7, entries=365319, sequenceid=262177331, filesize=8.2 M 2020-11-17 00:47:30,332 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.76 MB/106707307, heapSize ~128.08 MB/134305504, currentSize=2.26 MB/2365526 for e5dc5c620be19b02666a19c280ff582e in 879ms, sequenceid=262177331, compaction requested=true 2020-11-17 00:47:30,333 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 00:47:30,334 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7874e9fbeda34388bcbfc0487e635eb0, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6bf4e974e5e644af874bb7e8c4920173, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2217ecd985df43b7922d5bd83fdbb1b7] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=76.2 M 2020-11-17 00:47:30,368 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:30,370 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:30,396 INFO [Thread-464] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:33,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 66468 ns. 2020-11-17 00:47:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 183796 ns. 2020-11-17 00:47:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 41492 ns. 2020-11-17 00:47:42,189 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573966616 with entries=35919, filesize=128.56 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574062159 2020-11-17 00:47:44,268 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#91 average throughput is 45.48 MB/second, slept 30 time(s) and total slept time is 18636 ms. 1 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:47:44,330 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:44,362 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 676873b0804d4b408973fc841df5e7ea(size=93.4 M), total size for store is 10.9 G. This selection was in queue for 0sec, and took 33sec to execute. 2020-11-17 00:47:44,362 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605574030992; duration=33sec 2020-11-17 00:47:51,278 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:47:53,029 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#93 average throughput is 40.07 MB/second, slept 18 time(s) and total slept time is 11983 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:47:53,096 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 4d4a71e5b3d74be6b5e64fbe1fcfa9cf(size=77.9 M), total size for store is 48.4 G. This selection was in queue for 0sec, and took 22sec to execute. 2020-11-17 00:47:53,096 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605574050333; duration=22sec 2020-11-17 00:48:00,830 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:48:03,873 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:48:21,362 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:48:50,556 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=311.26 MB, max=1.83 GB, blockCount=39934, accesses=4653384, hits=4114839, hitRatio=88.43%, , cachingAccesses=4160291, cachingHits=4114012, cachingHitsRatio=98.89%, evictions=479, evicted=0, evictedPerRun=0.0 2020-11-17 00:49:50,671 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574062159 with entries=46658, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574190640 2020-11-17 00:49:52,826 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.19 MB heapSize=128.19 MB 2020-11-17 00:49:52,871 INFO [Thread-471] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:49:53,683 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.19 MB at sequenceid=258821675 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/b7e078d8d07644a9bb14bde972cb5e5e 2020-11-17 00:49:53,690 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:49:53,713 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/b7e078d8d07644a9bb14bde972cb5e5e, entries=377286, sequenceid=258821675, filesize=7.2 M 2020-11-17 00:49:53,716 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.19 MB/106110165, heapSize ~128.19 MB/134412336, currentSize=35.56 KB/36416 for 06f7ca9534710e37d90d9f8345864259 in 890ms, sequenceid=258821675, compaction requested=true 2020-11-17 00:49:53,717 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 00:49:53,718 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/3c66244b722647a78d4dda6b2cf1ee9e, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/f61e4665e57046f1bb51f09a7c993b60, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/b7e078d8d07644a9bb14bde972cb5e5e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=77.9 M 2020-11-17 00:49:53,755 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:49:53,758 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:49:53,784 INFO [Thread-473] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:50:11,428 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#95 average throughput is 57.29 MB/second, slept 18 time(s) and total slept time is 6422 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:50:11,533 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 95d4fa3500cd4a90b7ae1ed07c99ece2(size=78.2 M), total size for store is 55.2 G. This selection was in queue for 0sec, and took 17sec to execute. 2020-11-17 00:50:11,534 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605574193717; duration=17sec 2020-11-17 00:50:38,796 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:50:43,927 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.94 MB heapSize=128.68 MB 2020-11-17 00:50:43,971 INFO [Thread-477] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:50:44,651 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.94 MB at sequenceid=262186844 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/1259b3d627fb4d1183c361adc632bc10 2020-11-17 00:50:44,657 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:50:44,677 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1259b3d627fb4d1183c361adc632bc10, entries=343017, sequenceid=262186844, filesize=8.1 M 2020-11-17 00:50:44,678 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.94 MB/108984851, heapSize ~128.68 MB/134932416, currentSize=1.22 MB/1277322 for e5dc5c620be19b02666a19c280ff582e in 751ms, sequenceid=262186844, compaction requested=true 2020-11-17 00:50:48,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 17850 ns. 2020-11-17 00:50:48,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 70618 ns. 2020-11-17 00:51:03,951 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:51:21,605 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574190640 with entries=32974, filesize=128.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574281559 2020-11-17 00:51:30,123 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:51:30,480 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:51:37,444 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:51:39,434 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 68913 ns. 2020-11-17 00:51:39,434 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 95875 ns. 2020-11-17 00:53:18,058 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:53:24,825 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574281559 with entries=45295, filesize=128.08 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574404796 2020-11-17 00:53:38,769 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 67114 ns. 2020-11-17 00:53:38,769 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 161608 ns. 2020-11-17 00:53:38,770 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 45757 ns. 2020-11-17 00:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=312.40 MB, max=1.83 GB, blockCount=39905, accesses=4917862, hits=4362817, hitRatio=88.71%, , cachingAccesses=4408279, cachingHits=4361974, cachingHitsRatio=98.95%, evictions=509, evicted=0, evictedPerRun=0.0 2020-11-17 00:55:22,255 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574404796 with entries=43023, filesize=128.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574522224 2020-11-17 00:55:46,744 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.72 MB heapSize=128.04 MB 2020-11-17 00:55:46,780 INFO [Thread-490] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:55:47,555 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.72 MB at sequenceid=262201937 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/bd638f65ef8b43389a3426b216d396d8 2020-11-17 00:55:47,562 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:55:47,591 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/bd638f65ef8b43389a3426b216d396d8, entries=394375, sequenceid=262201937, filesize=8.2 M 2020-11-17 00:55:47,593 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.72 MB/104559287, heapSize ~128.04 MB/134262456, currentSize=267.66 KB/274085 for e5dc5c620be19b02666a19c280ff582e in 848ms, sequenceid=262201937, compaction requested=true 2020-11-17 00:55:47,594 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 00:55:47,594 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/4d4a71e5b3d74be6b5e64fbe1fcfa9cf, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1259b3d627fb4d1183c361adc632bc10, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/bd638f65ef8b43389a3426b216d396d8] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=94.2 M 2020-11-17 00:55:47,630 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:55:47,632 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:55:47,662 INFO [Thread-492] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:55:48,999 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15120 ns. 2020-11-17 00:55:49,000 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 47319 ns. 2020-11-17 00:56:06,951 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#98 average throughput is 57.56 MB/second, slept 21 time(s) and total slept time is 7642 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:56:07,034 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into f546e18fc9dc4227b092032697f3916d(size=95.6 M), total size for store is 48.4 G. This selection was in queue for 0sec, and took 19sec to execute. 2020-11-17 00:56:07,034 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605574547593; duration=19sec 2020-11-17 00:56:38,163 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:56:47,963 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:56:53,074 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574522224 with entries=33323, filesize=128.68 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574613034 2020-11-17 00:56:53,122 WARN [Close-WAL-Writer-47] asyncfs.FanOutOneBlockAsyncDFSOutputHelper: complete file /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574522224 not finished, retry = 0 2020-11-17 00:56:59,329 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:57:39,249 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 68294 ns. 2020-11-17 00:57:39,250 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 55794 ns. 2020-11-17 00:57:56,685 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:58:15,866 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.66 MB heapSize=128.13 MB 2020-11-17 00:58:15,913 INFO [Thread-500] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:58:16,629 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.66 MB at sequenceid=251397357 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/a8e030d3f84d478fa59a8e8c25a3feba 2020-11-17 00:58:16,636 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:58:16,654 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a8e030d3f84d478fa59a8e8c25a3feba, entries=320588, sequenceid=251397357, filesize=6.8 M 2020-11-17 00:58:16,657 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.66 MB/110796656, heapSize ~128.13 MB/134349640, currentSize=90.47 KB/92641 for 82421ce8c5c8b48c573baee83caa8305 in 791ms, sequenceid=251397357, compaction requested=true 2020-11-17 00:58:42,829 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:59:02,782 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574613034 with entries=47555, filesize=128.37 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574742741 2020-11-17 00:59:14,591 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.12 MB heapSize=128.00 MB 2020-11-17 00:59:14,633 INFO [Thread-505] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:59:15,739 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.12 MB at sequenceid=260696138 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/25f329345ce14ef59a54f2deabdb89df 2020-11-17 00:59:15,745 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:59:15,769 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/25f329345ce14ef59a54f2deabdb89df, entries=576210, sequenceid=260696138, filesize=8.8 M 2020-11-17 00:59:15,771 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.12 MB/89254745, heapSize ~128.00 MB/134221048, currentSize=135.32 KB/138569 for 73aae5715542e1a5370f7540d2117abc in 1180ms, sequenceid=260696138, compaction requested=true 2020-11-17 00:59:15,771 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 00:59:15,772 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/26eb2bad19164edbad2c808cc5824c8c, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/c5b5d1fb0ac949c7ad1a4c08dbd84397, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/25f329345ce14ef59a54f2deabdb89df] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=125.1 M 2020-11-17 00:59:15,817 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:59:15,820 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:59:15,846 INFO [Thread-507] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:59:20,995 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.23 MB heapSize=128.18 MB 2020-11-17 00:59:21,029 INFO [Thread-509] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:59:21,914 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.23 MB at sequenceid=262212790 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/8a78b43df6e247599b12db3c89586542 2020-11-17 00:59:21,927 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:59:21,961 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8a78b43df6e247599b12db3c89586542, entries=402653, sequenceid=262212790, filesize=8.2 M 2020-11-17 00:59:21,963 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~99.23 MB/104051959, heapSize ~128.18 MB/134405808, currentSize=832.10 KB/852073 for e5dc5c620be19b02666a19c280ff582e in 968ms, sequenceid=262212790, compaction requested=true 2020-11-17 00:59:23,097 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.81 MB heapSize=128.00 MB 2020-11-17 00:59:23,124 INFO [Thread-511] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:59:23,831 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.81 MB at sequenceid=271039200 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/fa22d77e9cf44220a13193e7d398da5c 2020-11-17 00:59:23,868 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/fa22d77e9cf44220a13193e7d398da5c, entries=319531, sequenceid=271039200, filesize=8.7 M 2020-11-17 00:59:23,870 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.81 MB/109897970, heapSize ~128.00 MB/134218376, currentSize=685.32 KB/701770 for 0469ce862509756add4a8cdb00bc465c in 773ms, sequenceid=271039200, compaction requested=true 2020-11-17 00:59:31,986 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:59:31,988 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=312.36 MB, max=1.83 GB, blockCount=39905, accesses=4967229, hits=4375210, hitRatio=88.08%, , cachingAccesses=4420590, cachingHits=4374273, cachingHitsRatio=98.95%, evictions=539, evicted=0, evictedPerRun=0.0 2020-11-17 00:59:37,103 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#101 average throughput is 56.34 MB/second, slept 18 time(s) and total slept time is 3733 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 00:59:37,155 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 00:59:37,176 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 0e49be7810c748eebaaa9369754cd54d(size=125.6 M), total size for store is 12.4 G. This selection was in queue for 0sec, and took 21sec to execute. 2020-11-17 00:59:37,176 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605574755771; duration=21sec 2020-11-17 00:59:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 156948 ns. 2020-11-17 00:59:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 52367 ns. 2020-11-17 00:59:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 65724 ns. 2020-11-17 00:59:49,193 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:00:23,424 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.16 MB heapSize=128.20 MB 2020-11-17 01:00:23,452 INFO [Thread-515] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:00:24,216 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.16 MB at sequenceid=272371696 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/f9f88df41d1349f29dfa9d52646bfe58 2020-11-17 01:00:24,222 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:00:24,242 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/f9f88df41d1349f29dfa9d52646bfe58, entries=274898, sequenceid=272371696, filesize=8.5 M 2020-11-17 01:00:24,243 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.16 MB/113409846, heapSize ~128.20 MB/134429328, currentSize=347.75 KB/356098 for fc93f6dcc564341dcc983bf30971f73d in 819ms, sequenceid=272371696, compaction requested=true 2020-11-17 01:00:24,244 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 01:00:24,244 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/dc5d10802b5340d7b6f98e15d4a31971, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/328927fa96354fc297b0108f0f8ebd3c, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/f9f88df41d1349f29dfa9d52646bfe58] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=59.2 M 2020-11-17 01:00:24,282 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:00:24,287 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:00:24,311 INFO [Thread-517] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:00:27,113 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574742741 with entries=32535, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574827090 2020-11-17 01:00:37,412 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#105 average throughput is 56.68 MB/second, slept 14 time(s) and total slept time is 6367 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 01:00:37,511 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into 307642a122394c60a0058a63b3afba66(size=59.9 M), total size for store is 33.2 G. This selection was in queue for 0sec, and took 13sec to execute. 2020-11-17 01:00:37,511 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605574824244; duration=13sec 2020-11-17 01:00:49,401 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18988 ns. 2020-11-17 01:00:49,402 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 48822 ns. 2020-11-17 01:02:01,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:02:06,023 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:02:33,052 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574827090 with entries=46960, filesize=128.54 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574953009 2020-11-17 01:02:44,834 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.63 MB heapSize=128.29 MB 2020-11-17 01:02:44,882 INFO [Thread-526] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:02:45,682 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.63 MB at sequenceid=262223171 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/69f194e3d47d4afeac90f59221f2b7a3 2020-11-17 01:02:45,687 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:02:45,707 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/69f194e3d47d4afeac90f59221f2b7a3, entries=356074, sequenceid=262223171, filesize=8.2 M 2020-11-17 01:02:45,708 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.63 MB/107613932, heapSize ~128.29 MB/134521888, currentSize=379.42 KB/388531 for e5dc5c620be19b02666a19c280ff582e in 874ms, sequenceid=262223171, compaction requested=true 2020-11-17 01:02:45,709 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 01:02:45,709 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f546e18fc9dc4227b092032697f3916d, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8a78b43df6e247599b12db3c89586542, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/69f194e3d47d4afeac90f59221f2b7a3] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=112.0 M 2020-11-17 01:02:45,741 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:02:45,742 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:02:45,768 INFO [Thread-528] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:03:05,365 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:03:08,456 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#107 average throughput is 57.83 MB/second, slept 26 time(s) and total slept time is 9583 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 01:03:08,555 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 240e861d28614dc783ef8b822036d576(size=113.6 M), total size for store is 48.4 G. This selection was in queue for 0sec, and took 22sec to execute. 2020-11-17 01:03:08,555 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605574965708; duration=22sec 2020-11-17 01:03:18,046 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:03:21,771 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:03:38,761 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 58673 ns. 2020-11-17 01:03:38,762 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 64644 ns. 2020-11-17 01:03:40,929 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:04:01,643 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:04:29,293 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574953009 with entries=42701, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575069251 2020-11-17 01:04:29,998 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.47 MB heapSize=128.00 MB 2020-11-17 01:04:30,027 INFO [Thread-534] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:04:31,052 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.47 MB at sequenceid=286491125 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/3aeaf603f1c441b6a40f7da5c21ea676 2020-11-17 01:04:31,059 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:04:31,083 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/3aeaf603f1c441b6a40f7da5c21ea676, entries=524177, sequenceid=286491125, filesize=9.1 M 2020-11-17 01:04:31,086 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.47 MB/94860263, heapSize ~128.00 MB/134218928, currentSize=35.03 KB/35870 for 5e5f05205d7e40b0141cd977dd009ba1 in 1088ms, sequenceid=286491125, compaction requested=true 2020-11-17 01:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=315.36 MB, max=1.83 GB, blockCount=39813, accesses=5010480, hits=4384335, hitRatio=87.50%, , cachingAccesses=4429718, cachingHits=4383384, cachingHitsRatio=98.95%, evictions=569, evicted=0, evictedPerRun=0.0 2020-11-17 01:04:40,669 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:05:03,854 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.32 MB heapSize=128.00 MB 2020-11-17 01:05:03,892 INFO [Thread-538] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:05:04,885 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.32 MB at sequenceid=290398997 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/0db452d1f1a24bdc8389eced5f284c76 2020-11-17 01:05:04,892 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:05:04,915 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/0db452d1f1a24bdc8389eced5f284c76, entries=551277, sequenceid=290398997, filesize=8.9 M 2020-11-17 01:05:04,916 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~88.32 MB/92607955, heapSize ~128.00 MB/134217848, currentSize=29.96 KB/30675 for bdaa31a63b7f3d06d9909b7560333e79 in 1063ms, sequenceid=290398997, compaction requested=true 2020-11-17 01:05:08,781 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:05:39,701 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 59200 ns. 2020-11-17 01:05:39,702 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 179034 ns. 2020-11-17 01:05:39,702 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 65505 ns. 2020-11-17 01:05:58,718 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 44320 ns. 2020-11-17 01:05:58,718 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 17093 ns. 2020-11-17 01:06:02,570 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575069251 with entries=34217, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575162540 2020-11-17 01:06:02,571 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573127291 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605573127291 2020-11-17 01:06:02,578 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573205423 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605573205423 2020-11-17 01:06:02,585 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573304425 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605573304425 2020-11-17 01:06:02,593 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573367799 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605573367799 2020-11-17 01:06:02,598 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573468108 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605573468108 2020-11-17 01:06:02,603 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573524841 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605573524841 2020-11-17 01:06:02,608 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573622015 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605573622015 2020-11-17 01:06:02,614 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573682040 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605573682040 2020-11-17 01:06:02,619 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573745114 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605573745114 2020-11-17 01:06:02,624 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573860489 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605573860489 2020-11-17 01:06:02,629 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605573966616 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605573966616 2020-11-17 01:06:02,642 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574062159 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605574062159 2020-11-17 01:06:04,727 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.32 MB heapSize=128.17 MB 2020-11-17 01:06:04,762 INFO [Thread-543] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:06:05,463 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.32 MB at sequenceid=262233142 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/68ab081153ac40588efb65982b86f0a3 2020-11-17 01:06:05,471 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:06:05,490 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/68ab081153ac40588efb65982b86f0a3, entries=344406, sequenceid=262233142, filesize=8.2 M 2020-11-17 01:06:05,492 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.32 MB/108342736, heapSize ~128.17 MB/134391760, currentSize=663.23 KB/679146 for e5dc5c620be19b02666a19c280ff582e in 765ms, sequenceid=262233142, compaction requested=true 2020-11-17 01:06:46,514 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:06:54,183 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:07:30,348 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=98.38 MB heapSize=128.01 MB 2020-11-17 01:07:30,402 INFO [Thread-550] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:07:31,578 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.38 MB at sequenceid=258876806 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/7a9d382166db46458b8a7b208821a25e 2020-11-17 01:07:31,586 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:07:31,618 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/7a9d382166db46458b8a7b208821a25e, entries=400539, sequenceid=258876806, filesize=7.8 M 2020-11-17 01:07:31,631 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.38 MB/103156379, heapSize ~128.01 MB/134222976, currentSize=123.65 KB/126621 for 06f7ca9534710e37d90d9f8345864259 in 1284ms, sequenceid=258876806, compaction requested=true 2020-11-17 01:08:23,264 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575162540 with entries=52182, filesize=128.15 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575303224 2020-11-17 01:08:23,265 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574190640 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605574190640 2020-11-17 01:08:23,281 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574281559 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605574281559 2020-11-17 01:08:23,299 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574404796 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605574404796 2020-11-17 01:08:23,305 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574522224 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605574522224 2020-11-17 01:08:39,585 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 90638 ns. 2020-11-17 01:08:39,585 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 109426 ns. 2020-11-17 01:08:59,421 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:09:07,771 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=314.06 MB, max=1.83 GB, blockCount=39841, accesses=5272870, hits=4646697, hitRatio=88.12%, , cachingAccesses=4692108, cachingHits=4645746, cachingHitsRatio=99.01%, evictions=599, evicted=0, evictedPerRun=0.0 2020-11-17 01:10:13,629 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575303224 with entries=40752, filesize=128.27 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575413591 2020-11-17 01:10:58,755 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 64707 ns. 2020-11-17 01:10:58,755 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 17968 ns. 2020-11-17 01:11:00,587 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.04 MB heapSize=128.05 MB 2020-11-17 01:11:00,635 INFO [Thread-560] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:11:01,638 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.04 MB at sequenceid=262248059 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/b3be4771d03c44f998cc9d6bb5371832 2020-11-17 01:11:01,646 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:11:01,670 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b3be4771d03c44f998cc9d6bb5371832, entries=404019, sequenceid=262248059, filesize=8.0 M 2020-11-17 01:11:01,673 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.04 MB/103851168, heapSize ~128.05 MB/134265224, currentSize=345.30 KB/353592 for e5dc5c620be19b02666a19c280ff582e in 1086ms, sequenceid=262248059, compaction requested=true 2020-11-17 01:11:39,420 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 69715 ns. 2020-11-17 01:11:39,420 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 206071 ns. 2020-11-17 01:11:39,420 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 60078 ns. 2020-11-17 01:11:51,971 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575413591 with entries=35847, filesize=128.18 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575511938 2020-11-17 01:11:57,036 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=104.98 MB heapSize=128.01 MB 2020-11-17 01:11:57,062 INFO [Thread-565] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:11:57,896 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.98 MB at sequenceid=251421468 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/834ad96d17864e82a83f33b6bd770e5e 2020-11-17 01:11:57,911 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:11:57,930 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/834ad96d17864e82a83f33b6bd770e5e, entries=328011, sequenceid=251421468, filesize=6.9 M 2020-11-17 01:11:57,932 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.98 MB/110079610, heapSize ~128.01 MB/134224160, currentSize=141.54 KB/144935 for 82421ce8c5c8b48c573baee83caa8305 in 897ms, sequenceid=251421468, compaction requested=true 2020-11-17 01:11:57,934 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 01:11:57,934 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/12afdd95bf214084aea6cab43115b9c6, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/676873b0804d4b408973fc841df5e7ea, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a8e030d3f84d478fa59a8e8c25a3feba, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/834ad96d17864e82a83f33b6bd770e5e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=233.0 M 2020-11-17 01:11:57,985 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:11:57,989 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:11:57,993 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:11:58,017 INFO [Thread-567] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:11:59,289 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:11:59,292 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:05,273 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:29,258 INFO [DataStreamer for file /hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/730679d52d49453e845a70db9bcd47b2] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:30,708 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:30,710 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:30,712 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:30,814 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:33,448 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:33,450 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:33,452 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:33,458 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:42,142 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:42,144 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:42,929 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:42,930 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:43,832 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:43,835 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:43,940 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:45,639 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:45,641 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:45,643 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:45,646 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:47,530 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.13 MB heapSize=128.22 MB 2020-11-17 01:12:47,563 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:47,567 INFO [Thread-573] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:48,354 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.13 MB at sequenceid=271078499 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/0098631f466947688ead05f671e8efa6 2020-11-17 01:12:48,365 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:48,429 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/0098631f466947688ead05f671e8efa6, entries=317227, sequenceid=271078499, filesize=8.7 M 2020-11-17 01:12:48,431 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.13 MB/110239454, heapSize ~128.22 MB/134448488, currentSize=583.40 KB/597406 for 0469ce862509756add4a8cdb00bc465c in 901ms, sequenceid=271078499, compaction requested=true 2020-11-17 01:12:48,433 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 01:12:48,433 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/e9102b0b74ce4a15a38c482b8bd2b94b, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/fa22d77e9cf44220a13193e7d398da5c, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/0098631f466947688ead05f671e8efa6] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=77.1 M 2020-11-17 01:12:48,533 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:48,540 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:48,594 INFO [Thread-575] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:49,254 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:12:50,892 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#114 average throughput is 55.50 MB/second, slept 52 time(s) and total slept time is 22204 ms. 1 active operations remaining, total limit is 57.69 MB/second 2020-11-17 01:12:51,009 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 730679d52d49453e845a70db9bcd47b2(size=229.4 M), total size for store is 10.9 G. This selection was in queue for 0sec, and took 53sec to execute. 2020-11-17 01:12:51,009 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605575517933; duration=53sec 2020-11-17 01:12:58,730 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:13:03,091 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:13:03,956 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:13:05,574 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#116 average throughput is 53.16 MB/second, slept 16 time(s) and total slept time is 8117 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 01:13:05,623 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:13:05,642 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 55e2815ec7bc491293eaa4bf847706c3(size=77.9 M), total size for store is 42.6 G. This selection was in queue for 0sec, and took 17sec to execute. 2020-11-17 01:13:05,642 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605575568432; duration=17sec 2020-11-17 01:13:16,792 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:13:21,041 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:13:31,907 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:13:41,034 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:13:58,340 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:14:09,441 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575511938 with entries=50134, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575649393 2020-11-17 01:14:09,442 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574613034 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605574613034 2020-11-17 01:14:12,062 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:14:34,277 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.81 MB heapSize=128.03 MB 2020-11-17 01:14:34,316 INFO [Thread-582] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:14:35,127 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.81 MB at sequenceid=262258967 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/3201dd6656824621b7c227f4ff95dc00 2020-11-17 01:14:35,134 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:14:35,157 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3201dd6656824621b7c227f4ff95dc00, entries=392297, sequenceid=262258967, filesize=8.2 M 2020-11-17 01:14:35,158 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~99.81 MB/104657926, heapSize ~128.03 MB/134250768, currentSize=703.69 KB/720577 for e5dc5c620be19b02666a19c280ff582e in 881ms, sequenceid=262258967, compaction requested=true 2020-11-17 01:14:35,159 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 01:14:35,159 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/68ab081153ac40588efb65982b86f0a3, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b3be4771d03c44f998cc9d6bb5371832, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3201dd6656824621b7c227f4ff95dc00] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.4 M 2020-11-17 01:14:35,197 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:14:35,199 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:14:35,226 INFO [Thread-584] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=315.78 MB, max=1.83 GB, blockCount=39786, accesses=5345939, hits=4656632, hitRatio=87.11%, , cachingAccesses=4701992, cachingHits=4655597, cachingHitsRatio=99.01%, evictions=629, evicted=0, evictedPerRun=0.0 2020-11-17 01:14:38,658 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 77415 ns. 2020-11-17 01:14:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 63003 ns. 2020-11-17 01:14:40,332 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#118 average throughput is 58.97 MB/second, slept 6 time(s) and total slept time is 1587 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 01:14:40,388 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 42ea8f6a7c6544418de3cd84c173ab02(size=25.2 M), total size for store is 48.5 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 01:14:40,388 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605575675158; duration=5sec 2020-11-17 01:14:43,627 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:15:09,340 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:15:17,804 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=86.13 MB heapSize=128.00 MB 2020-11-17 01:15:17,845 INFO [Thread-587] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:15:18,902 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=86.13 MB at sequenceid=260745962 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/9a71ba79298d4ab8b073f721639296c0 2020-11-17 01:15:18,909 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:15:18,925 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/9a71ba79298d4ab8b073f721639296c0, entries=572917, sequenceid=260745962, filesize=9.1 M 2020-11-17 01:15:18,927 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~86.13 MB/90311779, heapSize ~128.00 MB/134218720, currentSize=37.77 KB/38681 for 73aae5715542e1a5370f7540d2117abc in 1123ms, sequenceid=260745962, compaction requested=true 2020-11-17 01:15:31,100 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575649393 with entries=31905, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575731056 2020-11-17 01:15:41,826 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:15:47,786 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.05 MB heapSize=128.14 MB 2020-11-17 01:15:47,812 INFO [Thread-590] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:15:48,521 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.05 MB at sequenceid=272399968 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/3fd618a7608c480b9bf01ac2a4ae401b 2020-11-17 01:15:48,527 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:15:48,547 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/3fd618a7608c480b9bf01ac2a4ae401b, entries=275358, sequenceid=272399968, filesize=8.5 M 2020-11-17 01:15:48,550 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.05 MB/113302786, heapSize ~128.14 MB/134360944, currentSize=17.03 KB/17442 for fc93f6dcc564341dcc983bf30971f73d in 764ms, sequenceid=272399968, compaction requested=true 2020-11-17 01:16:06,896 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:16:08,769 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 27183 ns. 2020-11-17 01:16:08,770 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15466 ns. 2020-11-17 01:16:41,244 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:16:46,807 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:16:50,020 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:16:56,557 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:17:08,647 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:17:23,553 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:17:28,399 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575731056 with entries=43036, filesize=128.13 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575848365 2020-11-17 01:17:28,399 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574742741 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605574742741 2020-11-17 01:17:28,409 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574827090 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605574827090 2020-11-17 01:17:28,419 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605574953009 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605574953009 2020-11-17 01:17:29,692 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:17:39,506 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 65971 ns. 2020-11-17 01:17:39,507 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 58029 ns. 2020-11-17 01:17:39,507 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 167366 ns. 2020-11-17 01:17:42,758 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:17:50,983 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:17:57,502 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.95 MB heapSize=128.04 MB 2020-11-17 01:17:57,553 INFO [Thread-599] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:17:58,463 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.95 MB at sequenceid=262269191 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a3f2ccb1707a46bd8af4e5ab55844e54 2020-11-17 01:17:58,471 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:17:58,492 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a3f2ccb1707a46bd8af4e5ab55844e54, entries=347674, sequenceid=262269191, filesize=8.2 M 2020-11-17 01:17:58,494 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.95 MB/107954972, heapSize ~128.04 MB/134257320, currentSize=891.95 KB/913355 for e5dc5c620be19b02666a19c280ff582e in 992ms, sequenceid=262269191, compaction requested=true 2020-11-17 01:18:09,093 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:18:15,187 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:18:23,917 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:18:27,720 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:18:36,571 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:19:10,790 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575848365 with entries=36909, filesize=128.54 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575950713 2020-11-17 01:19:13,884 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=310.08 MB, max=1.83 GB, blockCount=39971, accesses=6086406, hits=5392778, hitRatio=88.60%, , cachingAccesses=5438350, cachingHits=5391742, cachingHitsRatio=99.14%, evictions=659, evicted=0, evictedPerRun=0.0 2020-11-17 01:19:38,887 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 72928 ns. 2020-11-17 01:19:38,888 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 95050 ns. 2020-11-17 01:21:14,637 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575950713 with entries=46456, filesize=128.28 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576074594 2020-11-17 01:21:18,919 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 25904 ns. 2020-11-17 01:21:18,919 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 39570 ns. 2020-11-17 01:22:06,975 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.02 MB heapSize=128.38 MB 2020-11-17 01:22:07,014 INFO [Thread-611] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:22:07,749 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.02 MB at sequenceid=262281638 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/c55d8b6eb7b949f69eb44db6611a8499 2020-11-17 01:22:07,757 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:22:07,791 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c55d8b6eb7b949f69eb44db6611a8499, entries=352070, sequenceid=262281638, filesize=8.3 M 2020-11-17 01:22:07,794 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~103.02 MB/108020196, heapSize ~128.38 MB/134615216, currentSize=1.22 MB/1275875 for e5dc5c620be19b02666a19c280ff582e in 819ms, sequenceid=262281638, compaction requested=true 2020-11-17 01:22:07,795 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 01:22:07,795 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/42ea8f6a7c6544418de3cd84c173ab02, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a3f2ccb1707a46bd8af4e5ab55844e54, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c55d8b6eb7b949f69eb44db6611a8499] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.7 M 2020-11-17 01:22:07,829 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:22:07,832 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:22:07,868 INFO [Thread-613] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:22:09,276 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:22:16,594 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#123 average throughput is 58.08 MB/second, slept 10 time(s) and total slept time is 3246 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 01:22:16,679 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 93d26d7c0ddb489e84d9214f31792e67(size=43.0 M), total size for store is 48.5 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 01:22:16,680 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605576127794; duration=8sec 2020-11-17 01:22:25,782 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:22:38,767 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:23:13,709 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:23:37,149 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576074594 with entries=52089, filesize=129.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576217018 2020-11-17 01:23:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 50598 ns. 2020-11-17 01:23:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 155844 ns. 2020-11-17 01:23:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 60549 ns. 2020-11-17 01:23:56,053 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:24:25,036 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:24:33,412 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=310.13 MB, max=1.83 GB, blockCount=39971, accesses=6333482, hits=5631599, hitRatio=88.92%, , cachingAccesses=5677175, cachingHits=5630555, cachingHitsRatio=99.18%, evictions=689, evicted=0, evictedPerRun=0.0 2020-11-17 01:24:39,525 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 84136 ns. 2020-11-17 01:24:39,526 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 57449 ns. 2020-11-17 01:25:55,481 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576217018 with entries=50320, filesize=128.56 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576355434 2020-11-17 01:26:20,182 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:26:24,467 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.32 MB heapSize=128.01 MB 2020-11-17 01:26:24,505 INFO [Thread-623] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:26:25,548 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.32 MB at sequenceid=262294439 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/4210093a4c8b41cc954574b4be368d38 2020-11-17 01:26:25,554 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:26:25,575 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/4210093a4c8b41cc954574b4be368d38, entries=413669, sequenceid=262294439, filesize=8.0 M 2020-11-17 01:26:25,577 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~98.32 MB/103093099, heapSize ~128.01 MB/134224592, currentSize=381.92 KB/391084 for e5dc5c620be19b02666a19c280ff582e in 1111ms, sequenceid=262294439, compaction requested=true 2020-11-17 01:26:28,727 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 24185 ns. 2020-11-17 01:26:28,727 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 31652 ns. 2020-11-17 01:26:28,953 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.39 MB heapSize=128.62 MB 2020-11-17 01:26:28,981 INFO [Thread-625] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:26:29,753 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.39 MB at sequenceid=271118716 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/9abc518666394cf0afa8f48fdbdefac2 2020-11-17 01:26:29,760 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:26:29,779 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/9abc518666394cf0afa8f48fdbdefac2, entries=319275, sequenceid=271118716, filesize=8.7 M 2020-11-17 01:26:29,780 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.39 MB/110512964, heapSize ~128.62 MB/134867232, currentSize=18.76 KB/19208 for 0469ce862509756add4a8cdb00bc465c in 827ms, sequenceid=271118716, compaction requested=true 2020-11-17 01:26:42,410 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=97.66 MB heapSize=128.00 MB 2020-11-17 01:26:42,446 INFO [Thread-629] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:26:43,250 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.66 MB at sequenceid=258936785 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/1a96d668136249348cb131f0946f17d0 2020-11-17 01:26:43,256 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:26:43,275 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/1a96d668136249348cb131f0946f17d0, entries=413434, sequenceid=258936785, filesize=8.0 M 2020-11-17 01:26:43,276 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~97.66 MB/102404608, heapSize ~128.00 MB/134218496, currentSize=152.25 KB/155903 for 06f7ca9534710e37d90d9f8345864259 in 866ms, sequenceid=258936785, compaction requested=true 2020-11-17 01:26:43,277 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 01:26:43,277 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/95d4fa3500cd4a90b7ae1ed07c99ece2, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/7a9d382166db46458b8a7b208821a25e, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/1a96d668136249348cb131f0946f17d0] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=93.9 M 2020-11-17 01:26:43,318 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:26:43,322 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:26:43,354 INFO [Thread-631] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:26:47,973 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:26:53,906 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:26:53,908 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:27:04,409 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#127 average throughput is 57.08 MB/second, slept 20 time(s) and total slept time is 7719 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 01:27:04,497 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into aaf65ae4fd62475282f096499989f64c(size=94.2 M), total size for store is 55.3 G. This selection was in queue for 0sec, and took 21sec to execute. 2020-11-17 01:27:04,497 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605576403276; duration=21sec 2020-11-17 01:27:20,312 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576355434 with entries=31517, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576440274 2020-11-17 01:27:34,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:27:52,664 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=104.41 MB heapSize=128.01 MB 2020-11-17 01:27:52,709 INFO [Thread-636] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:27:53,433 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.41 MB at sequenceid=251449081 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/ddb0e692e91f498d94eb59bcf113493e 2020-11-17 01:27:53,439 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:27:53,460 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/ddb0e692e91f498d94eb59bcf113493e, entries=335656, sequenceid=251449081, filesize=7.0 M 2020-11-17 01:27:53,462 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.41 MB/109487030, heapSize ~128.01 MB/134224048, currentSize=143.57 KB/147016 for 82421ce8c5c8b48c573baee83caa8305 in 798ms, sequenceid=251449081, compaction requested=true 2020-11-17 01:28:05,038 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:28:35,073 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:29:17,830 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.60 MB heapSize=128.01 MB 2020-11-17 01:29:17,870 INFO [Thread-641] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:29:18,788 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.60 MB at sequenceid=286572162 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/6871ddf84e5146559658af0b6961eb64 2020-11-17 01:29:18,796 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:29:18,820 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/6871ddf84e5146559658af0b6961eb64, entries=522295, sequenceid=286572162, filesize=9.2 M 2020-11-17 01:29:18,821 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.60 MB/95001880, heapSize ~128.01 MB/134231312, currentSize=380.86 KB/389999 for 5e5f05205d7e40b0141cd977dd009ba1 in 992ms, sequenceid=286572162, compaction requested=true 2020-11-17 01:29:18,822 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 01:29:18,822 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/62b5d05f3a254213a184a144e56cc2d5, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/3aeaf603f1c441b6a40f7da5c21ea676, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/6871ddf84e5146559658af0b6961eb64] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=79.3 M 2020-11-17 01:29:18,856 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:29:18,861 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:29:18,887 INFO [Thread-643] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:29:22,917 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:29:23,556 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:29:24,577 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576440274 with entries=46353, filesize=128.24 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576564553 2020-11-17 01:29:29,595 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:29:29,705 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:29:31,368 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:29:32,371 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#130 average throughput is 56.88 MB/second, slept 14 time(s) and total slept time is 3679 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 01:29:32,458 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into d4be82a8b9234318a59a49d00c0cfc1e(size=79.9 M), total size for store is 24.2 G. This selection was in queue for 0sec, and took 13sec to execute. 2020-11-17 01:29:32,458 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=10, startTime=1605576558822; duration=13sec 2020-11-17 01:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=310.72 MB, max=1.83 GB, blockCount=39959, accesses=6401828, hits=5667764, hitRatio=88.53%, , cachingAccesses=5713303, cachingHits=5666662, cachingHitsRatio=99.18%, evictions=719, evicted=0, evictedPerRun=0.0 2020-11-17 01:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 01:29:39,008 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 54811 ns. 2020-11-17 01:29:39,009 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 61436 ns. 2020-11-17 01:29:39,009 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 169318 ns. 2020-11-17 01:29:58,306 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.24 MB heapSize=128.27 MB 2020-11-17 01:29:58,346 INFO [Thread-646] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:29:59,081 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.24 MB at sequenceid=262305121 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/ec1602d968f342dcbbaec5bea58eb695 2020-11-17 01:29:59,090 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:29:59,114 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ec1602d968f342dcbbaec5bea58eb695, entries=346917, sequenceid=262305121, filesize=8.2 M 2020-11-17 01:29:59,115 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.24 MB/108257137, heapSize ~128.27 MB/134504784, currentSize=1.24 MB/1298192 for e5dc5c620be19b02666a19c280ff582e in 810ms, sequenceid=262305121, compaction requested=true 2020-11-17 01:29:59,116 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 01:29:59,116 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/93d26d7c0ddb489e84d9214f31792e67, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/4210093a4c8b41cc954574b4be368d38, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ec1602d968f342dcbbaec5bea58eb695] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=59.2 M 2020-11-17 01:29:59,152 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:29:59,154 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:29:59,181 INFO [Thread-648] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:30:11,401 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#132 average throughput is 57.99 MB/second, slept 14 time(s) and total slept time is 5036 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 01:30:11,532 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 5101895807fe494faa7563f8a0f98778(size=60.8 M), total size for store is 48.5 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 01:30:11,533 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605576599115; duration=12sec 2020-11-17 01:30:38,239 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576564553 with entries=27720, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576638197 2020-11-17 01:30:39,305 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 85761 ns. 2020-11-17 01:30:39,305 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 102146 ns. 2020-11-17 01:31:26,463 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.52 MB heapSize=128.66 MB 2020-11-17 01:31:26,490 INFO [Thread-653] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:31:27,182 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.52 MB at sequenceid=272428523 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/1245a07b0f4f49a28cc913fd38345645 2020-11-17 01:31:27,188 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:31:27,204 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/1245a07b0f4f49a28cc913fd38345645, entries=276202, sequenceid=272428523, filesize=8.5 M 2020-11-17 01:31:27,206 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.52 MB/113791262, heapSize ~128.66 MB/134914704, currentSize=46.31 KB/47425 for fc93f6dcc564341dcc983bf30971f73d in 742ms, sequenceid=272428523, compaction requested=true 2020-11-17 01:31:27,207 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 01:31:27,207 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/307642a122394c60a0058a63b3afba66, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/3fd618a7608c480b9bf01ac2a4ae401b, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/1245a07b0f4f49a28cc913fd38345645] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=76.9 M 2020-11-17 01:31:27,241 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:31:27,245 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:31:27,269 INFO [Thread-655] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:31:39,396 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 31144 ns. 2020-11-17 01:31:39,397 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14070 ns. 2020-11-17 01:31:43,747 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#134 average throughput is 58.20 MB/second, slept 18 time(s) and total slept time is 7877 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 01:31:43,788 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:31:43,810 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into 1d8b38fd470f42dc80a8584742c0f8bd(size=77.4 M), total size for store is 33.2 G. This selection was in queue for 0sec, and took 16sec to execute. 2020-11-17 01:31:43,810 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605576687206; duration=16sec 2020-11-17 01:31:47,392 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:32:22,439 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576638197 with entries=38559, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576742416 2020-11-17 01:32:25,644 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.76 MB heapSize=128.08 MB 2020-11-17 01:32:25,669 INFO [Thread-660] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:32:26,286 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.76 MB at sequenceid=251457240 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/87d20f5cf92145a6be6d32571912a9e8 2020-11-17 01:32:26,293 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:32:26,308 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/87d20f5cf92145a6be6d32571912a9e8, entries=305237, sequenceid=251457240, filesize=6.4 M 2020-11-17 01:32:26,311 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.76 MB/111944438, heapSize ~128.08 MB/134299696, currentSize=301.19 KB/308423 for 82421ce8c5c8b48c573baee83caa8305 in 667ms, sequenceid=251457240, compaction requested=true 2020-11-17 01:32:55,626 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.60 MB heapSize=128.02 MB 2020-11-17 01:32:55,674 INFO [Thread-663] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:32:56,803 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.60 MB at sequenceid=260800582 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/1416db1116234441b868d85f683867eb 2020-11-17 01:32:56,814 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:32:56,844 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/1416db1116234441b868d85f683867eb, entries=582382, sequenceid=260800582, filesize=9.1 M 2020-11-17 01:32:56,845 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.60 MB/89754033, heapSize ~128.02 MB/134234040, currentSize=145.85 KB/149355 for 73aae5715542e1a5370f7540d2117abc in 1220ms, sequenceid=260800582, compaction requested=true 2020-11-17 01:33:18,047 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:33:27,752 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:33:28,507 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576742416 with entries=24299, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576808473 2020-11-17 01:34:09,305 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.99 MB heapSize=128.08 MB 2020-11-17 01:34:09,345 INFO [Thread-669] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:34:10,091 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.99 MB at sequenceid=262317831 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/49ed2766b0bd462ab271e22cffff4204 2020-11-17 01:34:10,098 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:34:10,117 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/49ed2766b0bd462ab271e22cffff4204, entries=361836, sequenceid=262317831, filesize=8.1 M 2020-11-17 01:34:10,118 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.99 MB/106948122, heapSize ~128.08 MB/134299264, currentSize=749.85 KB/767848 for e5dc5c620be19b02666a19c280ff582e in 813ms, sequenceid=262317831, compaction requested=true 2020-11-17 01:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=312.63 MB, max=1.83 GB, blockCount=39908, accesses=6431108, hits=5670015, hitRatio=88.17%, , cachingAccesses=5715561, cachingHits=5668901, cachingHitsRatio=99.18%, evictions=749, evicted=0, evictedPerRun=0.0 2020-11-17 01:35:13,676 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576808473 with entries=39387, filesize=128.22 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576913636 2020-11-17 01:35:38,999 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 49397 ns. 2020-11-17 01:35:39,000 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 149744 ns. 2020-11-17 01:35:39,000 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 56568 ns. 2020-11-17 01:36:17,023 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576913636 with entries=23474, filesize=128.22 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576976984 2020-11-17 01:36:38,672 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 87673 ns. 2020-11-17 01:36:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 75066 ns. 2020-11-17 01:36:47,968 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:36:48,759 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 29454 ns. 2020-11-17 01:36:48,759 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 25485 ns. 2020-11-17 01:37:04,780 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.46 MB heapSize=128.00 MB 2020-11-17 01:37:04,818 INFO [Thread-678] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:37:05,409 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.46 MB at sequenceid=251465674 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/7a859a5b11364fb7b6ecfeebbff21b84 2020-11-17 01:37:05,417 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:37:05,437 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/7a859a5b11364fb7b6ecfeebbff21b84, entries=308111, sequenceid=251465674, filesize=6.3 M 2020-11-17 01:37:05,439 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.46 MB/111629659, heapSize ~128.00 MB/134218944, currentSize=13.42 KB/13747 for 82421ce8c5c8b48c573baee83caa8305 in 660ms, sequenceid=251465674, compaction requested=true 2020-11-17 01:37:05,441 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 01:37:05,441 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/ddb0e692e91f498d94eb59bcf113493e, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/87d20f5cf92145a6be6d32571912a9e8, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/7a859a5b11364fb7b6ecfeebbff21b84] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=19.7 M 2020-11-17 01:37:05,470 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:37:05,473 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:37:05,499 INFO [Thread-680] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:37:10,881 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#139 average throughput is 58.83 MB/second, slept 6 time(s) and total slept time is 2601 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 01:37:10,953 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 16c6db5febb8452a82161e2059612937(size=19.6 M), total size for store is 10.9 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 01:37:10,953 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605577025440; duration=5sec 2020-11-17 01:37:38,658 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:37:38,683 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.37 MB heapSize=128.06 MB 2020-11-17 01:37:38,724 INFO [Thread-683] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:37:39,530 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.37 MB at sequenceid=258971158 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/4d2167f3785548eb959166e5eb75e3ff 2020-11-17 01:37:39,558 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/4d2167f3785548eb959166e5eb75e3ff, entries=371546, sequenceid=258971158, filesize=7.1 M 2020-11-17 01:37:39,560 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.37 MB/106298283, heapSize ~128.06 MB/134281584, currentSize=189.30 KB/193848 for 06f7ca9534710e37d90d9f8345864259 in 877ms, sequenceid=258971158, compaction requested=true 2020-11-17 01:38:13,043 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576976984 with entries=42920, filesize=128.43 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577092991 2020-11-17 01:38:22,454 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=97.67 MB heapSize=128.04 MB 2020-11-17 01:38:22,490 INFO [Thread-686] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:23,518 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.67 MB at sequenceid=262330629 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/5aaf75085c2543589f770154318e48b9 2020-11-17 01:38:23,528 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:23,551 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5aaf75085c2543589f770154318e48b9, entries=422853, sequenceid=262330629, filesize=8.0 M 2020-11-17 01:38:23,559 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~97.67 MB/102414880, heapSize ~128.04 MB/134259656, currentSize=561.08 KB/574543 for e5dc5c620be19b02666a19c280ff582e in 1105ms, sequenceid=262330629, compaction requested=true 2020-11-17 01:38:23,560 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 01:38:23,560 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5101895807fe494faa7563f8a0f98778, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/49ed2766b0bd462ab271e22cffff4204, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5aaf75085c2543589f770154318e48b9] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=76.9 M 2020-11-17 01:38:23,589 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:23,591 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:23,628 INFO [Thread-688] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:37,413 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.76 MB heapSize=128.01 MB 2020-11-17 01:38:37,469 INFO [Thread-691] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:39,071 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.76 MB at sequenceid=290504515 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/85666e338c75472c8c4acd7db4d9448f 2020-11-17 01:38:39,077 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:39,097 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/85666e338c75472c8c4acd7db4d9448f, entries=545781, sequenceid=290504515, filesize=9.1 M 2020-11-17 01:38:39,100 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~88.76 MB/93068718, heapSize ~128.01 MB/134223480, currentSize=178.35 KB/182635 for bdaa31a63b7f3d06d9909b7560333e79 in 1688ms, sequenceid=290504515, compaction requested=true 2020-11-17 01:38:39,101 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-17 01:38:39,101 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/d5709fce1e79448d93769d4871586dcd, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/0db452d1f1a24bdc8389eced5f284c76, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/85666e338c75472c8c4acd7db4d9448f] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp, totalSize=67.2 M 2020-11-17 01:38:39,173 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:39,176 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:39,203 INFO [Thread-693] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:39,473 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#142 average throughput is 57.30 MB/second, slept 17 time(s) and total slept time is 5086 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 01:38:39,549 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:39,570 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into f81a671a601f454699aaa03c534fc8f2(size=78.3 M), total size for store is 48.5 G. This selection was in queue for 0sec, and took 16sec to execute. 2020-11-17 01:38:39,570 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605577103559; duration=16sec 2020-11-17 01:38:45,486 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:46,508 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:48,372 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:48,458 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:49,104 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:38:51,016 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: bdaa31a63b7f3d06d9909b7560333e79#e#compaction#144 average throughput is 56.24 MB/second, slept 9 time(s) and total slept time is 1267 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 01:38:51,131 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of bdaa31a63b7f3d06d9909b7560333e79 into 3c4cbc976ee74d0798e8aac9f64d31a7(size=66.9 M), total size for store is 31.8 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 01:38:51,131 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., storeName=e, priority=11, startTime=1605577119100; duration=12sec 2020-11-17 01:38:54,972 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:39:26,128 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577092991 with entries=26585, filesize=128.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577166084 2020-11-17 01:39:26,128 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575069251 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605575069251 2020-11-17 01:39:26,144 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575162540 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605575162540 2020-11-17 01:39:26,152 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575303224 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605575303224 2020-11-17 01:39:26,158 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575413591 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605575413591 2020-11-17 01:39:26,165 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575511938 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605575511938 2020-11-17 01:39:26,172 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575649393 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605575649393 2020-11-17 01:39:26,177 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575731056 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605575731056 2020-11-17 01:39:26,182 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575848365 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605575848365 2020-11-17 01:39:26,188 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605575950713 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605575950713 2020-11-17 01:39:26,193 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576074594 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605576074594 2020-11-17 01:39:26,199 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576217018 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605576217018 2020-11-17 01:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=311.66 MB, max=1.83 GB, blockCount=39925, accesses=6608912, hits=5817023, hitRatio=88.02%, , cachingAccesses=5862504, cachingHits=5815826, cachingHitsRatio=99.20%, evictions=779, evicted=0, evictedPerRun=0.0 2020-11-17 01:39:52,071 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:40:04,552 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.93 MB heapSize=128.00 MB 2020-11-17 01:40:04,602 INFO [Thread-699] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:40:05,465 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.93 MB at sequenceid=271158113 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/1f497bebaa8d4c25a6ad9e018b148acf 2020-11-17 01:40:05,471 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:40:05,493 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/1f497bebaa8d4c25a6ad9e018b148acf, entries=317043, sequenceid=271158113, filesize=8.7 M 2020-11-17 01:40:05,494 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.93 MB/110029696, heapSize ~128.00 MB/134218312, currentSize=24.31 KB/24892 for 0469ce862509756add4a8cdb00bc465c in 943ms, sequenceid=271158113, compaction requested=true 2020-11-17 01:40:05,495 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 01:40:05,496 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/55e2815ec7bc491293eaa4bf847706c3, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/9abc518666394cf0afa8f48fdbdefac2, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/1f497bebaa8d4c25a6ad9e018b148acf] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=95.3 M 2020-11-17 01:40:05,560 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:40:05,567 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:40:05,597 INFO [Thread-701] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:40:16,344 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:40:25,121 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:40:27,010 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#146 average throughput is 51.80 MB/second, slept 20 time(s) and total slept time is 7919 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 01:40:27,116 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 54cf077577e349d1bbc06f1ab6021bfd(size=95.9 M), total size for store is 42.6 G. This selection was in queue for 0sec, and took 21sec to execute. 2020-11-17 01:40:27,116 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605577205494; duration=21sec 2020-11-17 01:41:01,510 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:41:14,673 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577166084 with entries=39212, filesize=128.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577274627 2020-11-17 01:41:14,673 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576355434 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605576355434 2020-11-17 01:41:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 87571 ns. 2020-11-17 01:41:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 187467 ns. 2020-11-17 01:41:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 52204 ns. 2020-11-17 01:41:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 60775 ns. 2020-11-17 01:41:52,507 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.57 MB heapSize=128.10 MB 2020-11-17 01:41:52,558 INFO [Thread-708] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:41:53,384 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.57 MB at sequenceid=262340983 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/dce5f6aee12e4e61a7037f3f5f564e57 2020-11-17 01:41:53,394 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:41:53,420 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/dce5f6aee12e4e61a7037f3f5f564e57, entries=367911, sequenceid=262340983, filesize=8.3 M 2020-11-17 01:41:53,423 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.57 MB/106500391, heapSize ~128.10 MB/134319672, currentSize=981.79 KB/1005356 for e5dc5c620be19b02666a19c280ff582e in 916ms, sequenceid=262340983, compaction requested=true 2020-11-17 01:41:58,718 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14113 ns. 2020-11-17 01:41:58,718 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 73973 ns. 2020-11-17 01:42:26,801 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.59 MB heapSize=128.44 MB 2020-11-17 01:42:26,832 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577274627 with entries=26490, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577346802 2020-11-17 01:42:26,837 INFO [Thread-711] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:42:27,436 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.59 MB at sequenceid=251475043 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/b29b54507495459fa2b7514f2622e013 2020-11-17 01:42:27,442 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:42:27,459 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b29b54507495459fa2b7514f2622e013, entries=312344, sequenceid=251475043, filesize=6.4 M 2020-11-17 01:42:27,462 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.59 MB/111767405, heapSize ~128.44 MB/134678352, currentSize=12.67 KB/12976 for 82421ce8c5c8b48c573baee83caa8305 in 661ms, sequenceid=251475043, compaction requested=true 2020-11-17 01:42:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 73792 ns. 2020-11-17 01:43:01,534 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:43:18,047 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:44:23,448 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577346802 with entries=43096, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577463406 2020-11-17 01:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=315.14 MB, max=1.83 GB, blockCount=39823, accesses=6806272, hits=5996438, hitRatio=88.10%, , cachingAccesses=6041909, cachingHits=5995221, cachingHitsRatio=99.23%, evictions=809, evicted=0, evictedPerRun=0.0 2020-11-17 01:44:56,312 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:45:11,763 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:45:20,605 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.69 MB heapSize=128.17 MB 2020-11-17 01:45:20,645 INFO [Thread-719] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:45:21,397 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.69 MB at sequenceid=262351595 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/15dd0fe86ab5452d91ecd58dcc7a9619 2020-11-17 01:45:21,405 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:45:21,425 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/15dd0fe86ab5452d91ecd58dcc7a9619, entries=367353, sequenceid=262351595, filesize=8.2 M 2020-11-17 01:45:21,427 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.69 MB/106628026, heapSize ~128.17 MB/134394488, currentSize=740.37 KB/758141 for e5dc5c620be19b02666a19c280ff582e in 823ms, sequenceid=262351595, compaction requested=true 2020-11-17 01:45:21,428 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 01:45:21,431 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/240e861d28614dc783ef8b822036d576, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f81a671a601f454699aaa03c534fc8f2, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/dce5f6aee12e4e61a7037f3f5f564e57, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/15dd0fe86ab5452d91ecd58dcc7a9619] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=208.3 M 2020-11-17 01:45:21,474 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:45:21,479 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:45:21,482 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:45:21,509 INFO [Thread-721] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:45:26,491 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:45:28,040 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577463406 with entries=24892, filesize=128.36 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577528008 2020-11-17 01:45:35,128 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:45:46,697 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/0367aebceb5f4d13a159c58ab1ce66a2] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:45:59,744 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:46:03,639 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#150 average throughput is 57.54 MB/second, slept 42 time(s) and total slept time is 12609 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 01:46:03,749 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 0367aebceb5f4d13a159c58ab1ce66a2(size=211.7 M), total size for store is 48.5 G. This selection was in queue for 0sec, and took 42sec to execute. 2020-11-17 01:46:03,750 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605577521427; duration=42sec 2020-11-17 01:46:20,278 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:46:34,601 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:46:42,471 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:46:53,964 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.26 MB heapSize=128.61 MB 2020-11-17 01:46:53,999 INFO [Thread-729] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:46:54,746 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.26 MB at sequenceid=272456745 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/b5377764143242ecb9e81a4d85372dc4 2020-11-17 01:46:54,752 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:46:54,770 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/b5377764143242ecb9e81a4d85372dc4, entries=279320, sequenceid=272456745, filesize=8.5 M 2020-11-17 01:46:54,773 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.26 MB/113514366, heapSize ~128.61 MB/134852336, currentSize=23.67 KB/24233 for fc93f6dcc564341dcc983bf30971f73d in 809ms, sequenceid=272456745, compaction requested=true 2020-11-17 01:47:05,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:47:09,162 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 37014 ns. 2020-11-17 01:47:09,163 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15308 ns. 2020-11-17 01:47:15,894 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577528008 with entries=38797, filesize=128.60 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577635845 2020-11-17 01:47:18,299 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:47:25,799 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:47:32,584 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.44 MB heapSize=128.27 MB 2020-11-17 01:47:32,615 INFO [Thread-733] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:47:33,318 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.44 MB at sequenceid=251484087 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/6bb5bf6131884cde8cfec52a283a052e 2020-11-17 01:47:33,326 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:47:33,344 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/6bb5bf6131884cde8cfec52a283a052e, entries=311990, sequenceid=251484087, filesize=6.4 M 2020-11-17 01:47:33,346 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.44 MB/111612815, heapSize ~128.27 MB/134496776, currentSize=336.97 KB/345060 for 82421ce8c5c8b48c573baee83caa8305 in 761ms, sequenceid=251484087, compaction requested=true 2020-11-17 01:47:33,346 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 01:47:33,347 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/16c6db5febb8452a82161e2059612937, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b29b54507495459fa2b7514f2622e013, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/6bb5bf6131884cde8cfec52a283a052e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=32.4 M 2020-11-17 01:47:33,378 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:47:33,380 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:47:33,405 INFO [Thread-735] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:47:39,501 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 93532 ns. 2020-11-17 01:47:39,502 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 194864 ns. 2020-11-17 01:47:39,502 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 48949 ns. 2020-11-17 01:47:39,502 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 64855 ns. 2020-11-17 01:47:39,502 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 104614 ns. 2020-11-17 01:47:42,378 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#153 average throughput is 59.01 MB/second, slept 10 time(s) and total slept time is 4548 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 01:47:42,431 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into f76c7ef16cb248b3972fb397811a6ab2(size=32.3 M), total size for store is 10.9 G. This selection was in queue for 0sec, and took 9sec to execute. 2020-11-17 01:47:42,431 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605577653346; duration=9sec 2020-11-17 01:47:50,476 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:05,899 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.37 MB heapSize=128.00 MB 2020-11-17 01:48:05,947 INFO [Thread-738] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:06,612 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.37 MB at sequenceid=259003634 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/b5723b740b974527a31cb250de491fa1 2020-11-17 01:48:06,620 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:06,647 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/b5723b740b974527a31cb250de491fa1, entries=373593, sequenceid=259003634, filesize=7.1 M 2020-11-17 01:48:06,649 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.37 MB/106297829, heapSize ~128.00 MB/134218296, currentSize=191.14 KB/195727 for 06f7ca9534710e37d90d9f8345864259 in 750ms, sequenceid=259003634, compaction requested=true 2020-11-17 01:48:06,650 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 01:48:06,650 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/4c2528fece724d308da264632b00b6c6, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/aaf65ae4fd62475282f096499989f64c, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/4d2167f3785548eb959166e5eb75e3ff, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/b5723b740b974527a31cb250de491fa1] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=235.0 M 2020-11-17 01:48:06,699 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:06,702 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:06,718 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:06,729 INFO [Thread-740] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:13,294 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:16,952 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577635845 with entries=22865, filesize=128.27 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577696902 2020-11-17 01:48:27,602 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:27,607 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:28,854 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:30,058 INFO [DataStreamer for file /hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/8a8094eba4da4ecdaea58b821b8c7dad] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:35,357 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:36,236 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:36,824 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:36,826 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:36,828 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:40,228 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:40,230 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:40,231 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:40,233 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:53,809 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#155 average throughput is 57.98 MB/second, slept 49 time(s) and total slept time is 15731 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 01:48:53,853 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:48:53,872 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 8a8094eba4da4ecdaea58b821b8c7dad(size=232.3 M), total size for store is 55.3 G. This selection was in queue for 0sec, and took 47sec to execute. 2020-11-17 01:48:53,872 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605577686649; duration=47sec 2020-11-17 01:49:11,724 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:49:12,212 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:49:12,218 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:49:15,718 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:49:20,342 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=310.19 MB, max=1.83 GB, blockCount=39935, accesses=6970765, hits=6068364, hitRatio=87.05%, , cachingAccesses=6113796, cachingHits=6066967, cachingHitsRatio=99.23%, evictions=839, evicted=0, evictedPerRun=0.0 2020-11-17 01:49:52,714 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577696902 with entries=34984, filesize=128.28 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577792609 2020-11-17 01:49:54,069 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:49:56,524 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.15 MB heapSize=128.11 MB 2020-11-17 01:49:56,562 INFO [Thread-748] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:49:57,338 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.15 MB at sequenceid=262365246 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/bae0fe9efa0a40c091642d8dc37c6c62 2020-11-17 01:49:57,344 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:49:57,363 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/bae0fe9efa0a40c091642d8dc37c6c62, entries=374100, sequenceid=262365246, filesize=8.0 M 2020-11-17 01:49:57,365 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.15 MB/106064385, heapSize ~128.11 MB/134335168, currentSize=241.38 KB/247177 for e5dc5c620be19b02666a19c280ff582e in 841ms, sequenceid=262365246, compaction requested=true 2020-11-17 01:50:03,241 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:50:05,816 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.86 MB heapSize=128.03 MB 2020-11-17 01:50:05,849 INFO [Thread-750] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:50:06,993 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.86 MB at sequenceid=260853912 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/63a7b98180b646d592845af1fa8f55bb 2020-11-17 01:50:07,016 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/63a7b98180b646d592845af1fa8f55bb, entries=578110, sequenceid=260853912, filesize=9.1 M 2020-11-17 01:50:07,027 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.86 MB/90031449, heapSize ~128.03 MB/134245736, currentSize=52.16 KB/53410 for 73aae5715542e1a5370f7540d2117abc in 1211ms, sequenceid=260853912, compaction requested=true 2020-11-17 01:50:07,028 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 01:50:07,028 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/9a71ba79298d4ab8b073f721639296c0, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/1416db1116234441b868d85f683867eb, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/63a7b98180b646d592845af1fa8f55bb] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=27.2 M 2020-11-17 01:50:07,055 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:50:07,059 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:50:07,081 INFO [Thread-752] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:50:07,162 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:50:08,002 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:50:11,505 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#158 average throughput is 57.46 MB/second, slept 4 time(s) and total slept time is 524 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 01:50:11,576 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 3883ef3c9ab74dd79f03904c75341083(size=27.7 M), total size for store is 12.4 G. This selection was in queue for 0sec, and took 4sec to execute. 2020-11-17 01:50:11,576 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=10, startTime=1605577807027; duration=4sec 2020-11-17 01:50:17,801 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:50:40,002 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:50:46,593 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:51:14,769 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577792609 with entries=30375, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577874731 2020-11-17 01:51:18,529 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:51:23,646 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:51:32,643 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:51:40,100 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:51:44,668 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:51:52,868 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:51:55,625 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.51 MB heapSize=128.29 MB 2020-11-17 01:51:55,678 INFO [Thread-758] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:51:56,346 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.51 MB at sequenceid=251491933 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/ba58b99ea8384a0baa34b7153997de42 2020-11-17 01:51:56,352 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:51:56,370 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/ba58b99ea8384a0baa34b7153997de42, entries=311360, sequenceid=251491933, filesize=6.3 M 2020-11-17 01:51:56,372 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.51 MB/111679566, heapSize ~128.29 MB/134519200, currentSize=851.00 KB/871419 for 82421ce8c5c8b48c573baee83caa8305 in 747ms, sequenceid=251491933, compaction requested=true 2020-11-17 01:52:00,641 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:52:08,316 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:52:16,610 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:52:18,743 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 38771 ns. 2020-11-17 01:52:18,744 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 13259 ns. 2020-11-17 01:52:29,793 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:52:37,036 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:52:47,804 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577874731 with entries=33667, filesize=128.41 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577967754 2020-11-17 01:52:55,795 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.28 MB heapSize=128.03 MB 2020-11-17 01:52:55,828 INFO [Thread-762] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:52:56,870 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.28 MB at sequenceid=286649132 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/fa2aac9d60ff41979c57445c27fca5db 2020-11-17 01:52:56,876 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:52:56,893 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/fa2aac9d60ff41979c57445c27fca5db, entries=527378, sequenceid=286649132, filesize=9.1 M 2020-11-17 01:52:56,894 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.28 MB/94667346, heapSize ~128.03 MB/134244664, currentSize=52.62 KB/53887 for 5e5f05205d7e40b0141cd977dd009ba1 in 1099ms, sequenceid=286649132, compaction requested=true 2020-11-17 01:52:56,895 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 01:52:56,896 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/2820dbe5157547fe93aeb9ae0af68d46, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/af53171106a64244b5c3aa0336d0abfe, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/d4be82a8b9234318a59a49d00c0cfc1e, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/fa2aac9d60ff41979c57445c27fca5db] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=428.9 M 2020-11-17 01:52:56,955 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:52:56,958 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:52:56,964 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:52:56,993 INFO [Thread-764] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:01,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:08,798 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:16,944 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:17,419 INFO [DataStreamer for file /hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/a8f0e9bee2454273a10d903e757bc53b] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:18,055 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:18,079 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:18,198 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:18,310 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:18,879 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:18,996 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:19,112 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:19,246 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:22,442 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:22,448 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:22,594 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:22,787 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:22,894 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:23,010 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:23,094 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:23,177 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:23,181 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:23,274 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:24,349 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:24,351 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:24,949 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:36,501 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.53 MB heapSize=128.77 MB 2020-11-17 01:53:36,550 INFO [Thread-769] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:37,490 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.53 MB at sequenceid=271197235 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/175350c7dcec4783bba39fc3a1c01918 2020-11-17 01:53:37,497 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:37,515 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/175350c7dcec4783bba39fc3a1c01918, entries=319550, sequenceid=271197235, filesize=8.7 M 2020-11-17 01:53:37,522 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.53 MB/110652855, heapSize ~128.77 MB/135024960, currentSize=24.80 KB/25395 for 0469ce862509756add4a8cdb00bc465c in 1022ms, sequenceid=271197235, compaction requested=true 2020-11-17 01:53:39,042 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 106264 ns. 2020-11-17 01:53:39,042 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 93906 ns. 2020-11-17 01:53:39,042 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 55926 ns. 2020-11-17 01:53:39,043 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 169514 ns. 2020-11-17 01:53:39,043 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 68264 ns. 2020-11-17 01:53:40,203 INFO [DataStreamer for file /hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/a8f0e9bee2454273a10d903e757bc53b] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:43,444 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.03 MB heapSize=128.31 MB 2020-11-17 01:53:43,483 INFO [Thread-772] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:44,505 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.03 MB at sequenceid=262376587 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/bb1a16bea8bd4ed0bb55143e803d03d3 2020-11-17 01:53:44,510 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:44,541 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/bb1a16bea8bd4ed0bb55143e803d03d3, entries=421204, sequenceid=262376587, filesize=8.0 M 2020-11-17 01:53:44,544 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~98.03 MB/102796334, heapSize ~128.31 MB/134545704, currentSize=1.40 MB/1462780 for e5dc5c620be19b02666a19c280ff582e in 1101ms, sequenceid=262376587, compaction requested=true 2020-11-17 01:53:47,792 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:47,979 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:48,063 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:48,136 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:48,704 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:48,898 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:51,467 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:51,876 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:51,879 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:53,003 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:53,005 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:53,008 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:53,011 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:53,712 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:54,893 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:55,096 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:55,099 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:55,102 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:55,240 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:55,247 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:55,374 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:55,592 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:55,727 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:56,132 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:56,251 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:56,254 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:57,347 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:57,992 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:57,994 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:57,995 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:59,604 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:59,606 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:59,617 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:53:59,633 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:54:00,461 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:54:00,857 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:54:01,827 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577967754 with entries=27208, filesize=128.75 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578041773 2020-11-17 01:54:01,828 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576440274 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605576440274 2020-11-17 01:54:01,838 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576564553 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605576564553 2020-11-17 01:54:01,855 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576638197 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605576638197 2020-11-17 01:54:01,864 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576742416 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605576742416 2020-11-17 01:54:01,877 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576808473 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605576808473 2020-11-17 01:54:01,882 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576913636 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605576913636 2020-11-17 01:54:01,893 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605576976984 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605576976984 2020-11-17 01:54:02,881 INFO [DataStreamer for file /hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/a8f0e9bee2454273a10d903e757bc53b] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:54:04,204 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:54:06,314 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#161 average throughput is 55.04 MB/second, slept 59 time(s) and total slept time is 15747 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 01:54:06,434 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into a8f0e9bee2454273a10d903e757bc53b(size=409.9 M), total size for store is 24.2 G. This selection was in queue for 0sec, and took 1mins, 9sec to execute. 2020-11-17 01:54:06,434 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=11, startTime=1605577976895; duration=1mins, 9sec 2020-11-17 01:54:15,688 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:54:16,421 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.53 GB, freeSize=308.07 MB, max=1.83 GB, blockCount=39990, accesses=7783930, hits=6814602, hitRatio=87.55%, , cachingAccesses=6860056, cachingHits=6812947, cachingHitsRatio=99.31%, evictions=869, evicted=0, evictedPerRun=0.0 2020-11-17 01:54:45,896 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:55:05,662 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578041773 with entries=23321, filesize=128.66 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578105602 2020-11-17 01:55:06,402 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.62 MB heapSize=128.14 MB 2020-11-17 01:55:06,443 INFO [Thread-779] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:55:06,966 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.62 MB at sequenceid=251497677 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/3ced27ec8fe343aaba66e4b82f1faa12 2020-11-17 01:55:06,972 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:55:06,991 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/3ced27ec8fe343aaba66e4b82f1faa12, entries=307818, sequenceid=251497677, filesize=6.2 M 2020-11-17 01:55:06,992 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.62 MB/111794467, heapSize ~128.14 MB/134363408, currentSize=47.94 KB/49086 for 82421ce8c5c8b48c573baee83caa8305 in 591ms, sequenceid=251497677, compaction requested=true 2020-11-17 01:55:06,993 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 01:55:06,994 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/f76c7ef16cb248b3972fb397811a6ab2, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/ba58b99ea8384a0baa34b7153997de42, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/3ced27ec8fe343aaba66e4b82f1faa12] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=44.8 M 2020-11-17 01:55:07,036 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:55:07,038 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:55:07,064 INFO [Thread-781] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:55:14,811 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:55:18,232 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:55:19,705 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#165 average throughput is 58.74 MB/second, slept 14 time(s) and total slept time is 5837 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 01:55:19,813 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into fa2ba6b012c84d1190e8311273af22c1(size=44.8 M), total size for store is 10.9 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 01:55:19,813 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605578106993; duration=12sec 2020-11-17 01:56:28,333 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=102.06 MB heapSize=128.03 MB 2020-11-17 01:56:28,385 INFO [Thread-787] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:56:29,194 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.06 MB at sequenceid=259029637 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/f04d72b888f847598c29ff6a0a3d1dba 2020-11-17 01:56:29,201 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:56:29,221 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/f04d72b888f847598c29ff6a0a3d1dba, entries=363640, sequenceid=259029637, filesize=6.8 M 2020-11-17 01:56:29,223 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.06 MB/107019451, heapSize ~128.03 MB/134249528, currentSize=365.49 KB/374259 for 06f7ca9534710e37d90d9f8345864259 in 891ms, sequenceid=259029637, compaction requested=true 2020-11-17 01:56:31,443 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578105602 with entries=31888, filesize=128.17 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578191417 2020-11-17 01:56:36,093 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:56:47,981 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:57:00,242 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:57:11,603 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.65 MB heapSize=128.01 MB 2020-11-17 01:57:11,658 INFO [Thread-792] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:57:12,511 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.65 MB at sequenceid=262387114 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/f38abccab43c4b31b4ad8b5a958e7ba4 2020-11-17 01:57:12,517 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:57:12,538 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f38abccab43c4b31b4ad8b5a958e7ba4, entries=364899, sequenceid=262387114, filesize=8.1 M 2020-11-17 01:57:12,540 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.65 MB/106584580, heapSize ~128.01 MB/134224848, currentSize=821.37 KB/841079 for e5dc5c620be19b02666a19c280ff582e in 937ms, sequenceid=262387114, compaction requested=true 2020-11-17 01:57:12,541 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 01:57:12,541 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/bae0fe9efa0a40c091642d8dc37c6c62, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/bb1a16bea8bd4ed0bb55143e803d03d3, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f38abccab43c4b31b4ad8b5a958e7ba4] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.1 M 2020-11-17 01:57:12,580 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:57:12,584 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:57:12,615 INFO [Thread-794] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:57:17,980 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#168 average throughput is 55.83 MB/second, slept 5 time(s) and total slept time is 1422 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 01:57:18,056 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 72f8095384c440909da8d306c40b087a(size=24.9 M), total size for store is 48.6 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 01:57:18,056 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605578232540; duration=5sec 2020-11-17 01:57:18,765 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 20597 ns. 2020-11-17 01:57:18,765 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 30351 ns. 2020-11-17 01:57:27,906 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578191417 with entries=21426, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578247884 2020-11-17 01:57:56,768 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:58:10,752 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.45 MB heapSize=128.05 MB 2020-11-17 01:58:10,800 INFO [Thread-799] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:58:11,390 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.45 MB at sequenceid=251503262 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/a422415b0d5f4c1ea9c89978860498f3 2020-11-17 01:58:11,400 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:58:11,423 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a422415b0d5f4c1ea9c89978860498f3, entries=308852, sequenceid=251503262, filesize=6.2 M 2020-11-17 01:58:11,426 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.45 MB/111623126, heapSize ~128.05 MB/134272704, currentSize=355.08 KB/363606 for 82421ce8c5c8b48c573baee83caa8305 in 674ms, sequenceid=251503262, compaction requested=true 2020-11-17 01:58:32,795 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:59:02,865 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578247884 with entries=34486, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578342831 2020-11-17 01:59:10,473 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:59:24,102 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 01:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.52 GB, freeSize=317.38 MB, max=1.83 GB, blockCount=39732, accesses=7936314, hits=6950039, hitRatio=87.57%, , cachingAccesses=6995500, cachingHits=6948367, cachingHitsRatio=99.33%, evictions=899, evicted=0, evictedPerRun=0.0 2020-11-17 01:59:38,738 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 67397 ns. 2020-11-17 01:59:38,739 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 97052 ns. 2020-11-17 01:59:38,739 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 67369 ns. 2020-11-17 01:59:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 58813 ns. 2020-11-17 01:59:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 183686 ns. 2020-11-17 02:00:03,339 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:00:03,401 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:00:03,403 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:00:03,403 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:00:03,419 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:00:03,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:00:03,436 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:00:03,436 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:00:03,439 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:00:08,242 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578342831 with entries=24328, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578408212 2020-11-17 02:00:59,905 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.22 MB heapSize=128.00 MB 2020-11-17 02:00:59,950 INFO [Thread-809] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:01:00,821 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.22 MB at sequenceid=262398593 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/54640894dc1941d6b4ee614d5b99718e 2020-11-17 02:01:00,830 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:01:00,859 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/54640894dc1941d6b4ee614d5b99718e, entries=357177, sequenceid=262398593, filesize=8.1 M 2020-11-17 02:01:00,862 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.22 MB/107184178, heapSize ~128.00 MB/134219192, currentSize=44.63 KB/45700 for e5dc5c620be19b02666a19c280ff582e in 958ms, sequenceid=262398593, compaction requested=true 2020-11-17 02:01:46,680 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578408212 with entries=36881, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578506651 2020-11-17 02:02:01,464 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:02:05,963 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:02:15,000 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.06 MB heapSize=128.61 MB 2020-11-17 02:02:15,067 INFO [Thread-815] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:02:15,885 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.06 MB at sequenceid=272484823 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/73bfa9c892aa48cd8f0e4b2ba51c68b1 2020-11-17 02:02:15,890 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:02:15,907 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/73bfa9c892aa48cd8f0e4b2ba51c68b1, entries=282465, sequenceid=272484823, filesize=8.5 M 2020-11-17 02:02:15,910 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.06 MB/113304773, heapSize ~128.61 MB/134859656, currentSize=666.52 KB/682519 for fc93f6dcc564341dcc983bf30971f73d in 910ms, sequenceid=272484823, compaction requested=true 2020-11-17 02:02:15,911 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 02:02:15,911 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/1d8b38fd470f42dc80a8584742c0f8bd, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/b5377764143242ecb9e81a4d85372dc4, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/73bfa9c892aa48cd8f0e4b2ba51c68b1] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=94.4 M 2020-11-17 02:02:15,940 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:02:15,945 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:02:15,968 INFO [Thread-817] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:02:18,775 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 16203 ns. 2020-11-17 02:02:18,775 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 33820 ns. 2020-11-17 02:02:27,647 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.47 MB heapSize=128.00 MB 2020-11-17 02:02:27,678 INFO [Thread-819] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:02:28,362 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.47 MB at sequenceid=251511058 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/9bd09962ff9546ce8cfad77ea0560c4f 2020-11-17 02:02:28,369 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:02:28,390 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/9bd09962ff9546ce8cfad77ea0560c4f, entries=308083, sequenceid=251511058, filesize=6.3 M 2020-11-17 02:02:28,393 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.47 MB/111638061, heapSize ~128.00 MB/134217968, currentSize=276.43 KB/283066 for 82421ce8c5c8b48c573baee83caa8305 in 745ms, sequenceid=251511058, compaction requested=true 2020-11-17 02:02:28,394 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 02:02:28,394 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/fa2ba6b012c84d1190e8311273af22c1, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a422415b0d5f4c1ea9c89978860498f3, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/9bd09962ff9546ce8cfad77ea0560c4f] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=57.3 M 2020-11-17 02:02:28,429 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:02:28,434 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:02:28,460 INFO [Thread-821] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:02:36,947 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:02:40,409 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:02:40,412 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:02:43,925 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#172 average throughput is 41.93 MB/second, slept 22 time(s) and total slept time is 16534 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 02:02:44,031 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into 90cd25eb9111435c9d8adf7219d3fbec(size=94.7 M), total size for store is 33.2 G. This selection was in queue for 0sec, and took 28sec to execute. 2020-11-17 02:02:44,031 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605578535911; duration=28sec 2020-11-17 02:02:51,997 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#174 average throughput is 40.60 MB/second, slept 18 time(s) and total slept time is 13793 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 02:02:52,052 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into eb28451b27844712a385ceef797e951e(size=57.2 M), total size for store is 11.0 G. This selection was in queue for 0sec, and took 23sec to execute. 2020-11-17 02:02:52,052 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605578548393; duration=23sec 2020-11-17 02:03:16,658 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578506651 with entries=34192, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578596630 2020-11-17 02:03:22,309 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:03:41,724 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.65 GB, freeSize=186.02 MB, max=1.83 GB, blockCount=43605, accesses=8072708, hits=7048093, hitRatio=87.31%, , cachingAccesses=7097461, cachingHits=7046411, cachingHitsRatio=99.28%, evictions=929, evicted=0, evictedPerRun=0.0 2020-11-17 02:04:39,488 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 79959 ns. 2020-11-17 02:04:39,489 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 117508 ns. 2020-11-17 02:05:22,445 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578596630 with entries=46976, filesize=128.15 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578722417 2020-11-17 02:05:36,024 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.18 MB heapSize=128.13 MB 2020-11-17 02:05:36,070 INFO [Thread-832] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:05:36,911 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.18 MB at sequenceid=262412801 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/0b6d4495ae97455a9bfa161928724807 2020-11-17 02:05:36,919 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:05:36,938 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/0b6d4495ae97455a9bfa161928724807, entries=417249, sequenceid=262412801, filesize=8.1 M 2020-11-17 02:05:36,940 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.18 MB/102953447, heapSize ~128.13 MB/134357632, currentSize=440.17 KB/450739 for e5dc5c620be19b02666a19c280ff582e in 916ms, sequenceid=262412801, compaction requested=true 2020-11-17 02:05:36,941 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 02:05:36,941 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/72f8095384c440909da8d306c40b087a, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/54640894dc1941d6b4ee614d5b99718e, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/0b6d4495ae97455a9bfa161928724807] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.1 M 2020-11-17 02:05:36,968 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:05:36,971 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:05:36,996 INFO [Thread-834] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:05:38,750 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 91960 ns. 2020-11-17 02:05:38,750 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 173776 ns. 2020-11-17 02:05:38,750 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 66049 ns. 2020-11-17 02:05:45,240 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#176 average throughput is 60.68 MB/second, slept 10 time(s) and total slept time is 2918 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 02:05:45,289 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into b21d8fd5c4a94e4097a86b7d9d2f93af(size=42.4 M), total size for store is 48.6 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 02:05:45,289 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605578736940; duration=8sec 2020-11-17 02:06:45,603 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578722417 with entries=30919, filesize=128.34 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578805568 2020-11-17 02:06:47,969 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:06:56,113 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.92 MB heapSize=128.00 MB 2020-11-17 02:06:56,160 INFO [Thread-839] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:06:57,192 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.92 MB at sequenceid=260906229 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/9897a5cb004942a6a15d7a33e9d4ef39 2020-11-17 02:06:57,198 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:06:57,216 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/9897a5cb004942a6a15d7a33e9d4ef39, entries=581196, sequenceid=260906229, filesize=9.1 M 2020-11-17 02:06:57,218 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.92 MB/90088595, heapSize ~128.00 MB/134217752, currentSize=41.48 KB/42477 for 73aae5715542e1a5370f7540d2117abc in 1105ms, sequenceid=260906229, compaction requested=true 2020-11-17 02:07:06,845 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.00 MB heapSize=128.01 MB 2020-11-17 02:07:06,873 INFO [Thread-841] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:07:07,635 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.00 MB at sequenceid=259063253 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/bb69da67310341f6a6c28f260eac1acc 2020-11-17 02:07:07,641 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:07:07,672 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/bb69da67310341f6a6c28f260eac1acc, entries=371355, sequenceid=259063253, filesize=7.1 M 2020-11-17 02:07:07,678 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.00 MB/105901457, heapSize ~128.01 MB/134231968, currentSize=74.23 KB/76011 for 06f7ca9534710e37d90d9f8345864259 in 833ms, sequenceid=259063253, compaction requested=true 2020-11-17 02:07:16,562 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:07:28,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 23513 ns. 2020-11-17 02:07:28,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 33403 ns. 2020-11-17 02:07:29,116 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.06 MB heapSize=128.29 MB 2020-11-17 02:07:29,153 INFO [Thread-844] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:07:29,623 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:07:29,890 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.06 MB at sequenceid=271237702 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/3edb850a6a204383b3f7aea917b1c7fe 2020-11-17 02:07:29,911 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/3edb850a6a204383b3f7aea917b1c7fe, entries=319525, sequenceid=271237702, filesize=8.7 M 2020-11-17 02:07:29,913 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.06 MB/110160023, heapSize ~128.29 MB/134526768, currentSize=61.40 KB/62873 for 0469ce862509756add4a8cdb00bc465c in 798ms, sequenceid=271237702, compaction requested=true 2020-11-17 02:07:29,914 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 02:07:29,914 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/54cf077577e349d1bbc06f1ab6021bfd, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/175350c7dcec4783bba39fc3a1c01918, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/3edb850a6a204383b3f7aea917b1c7fe] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=113.3 M 2020-11-17 02:07:29,945 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:07:29,951 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:07:29,973 INFO [Thread-846] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:07:33,632 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=87.60 MB heapSize=128.00 MB 2020-11-17 02:07:33,667 INFO [Thread-848] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:07:34,750 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=87.60 MB at sequenceid=290595508 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/9ac91671065c4ab49adfdf584f83f42e 2020-11-17 02:07:34,756 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:07:34,887 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/9ac91671065c4ab49adfdf584f83f42e, entries=560447, sequenceid=290595508, filesize=8.6 M 2020-11-17 02:07:34,892 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~87.60 MB/91855009, heapSize ~128.00 MB/134217840, currentSize=27.28 KB/27930 for bdaa31a63b7f3d06d9909b7560333e79 in 1260ms, sequenceid=290595508, compaction requested=true 2020-11-17 02:07:47,079 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:07:49,606 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:07:52,057 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#180 average throughput is 59.57 MB/second, slept 23 time(s) and total slept time is 9862 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 02:07:52,132 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 7a682d1c43d44e0daf21aaf6bce0601f(size=113.8 M), total size for store is 42.6 G. This selection was in queue for 0sec, and took 22sec to execute. 2020-11-17 02:07:52,132 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605578849913; duration=22sec 2020-11-17 02:08:24,902 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:08:32,933 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:08:38,497 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:08:42,983 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:08:58,403 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578805568 with entries=49261, filesize=128.59 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578938356 2020-11-17 02:08:58,404 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577092991 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605577092991 2020-11-17 02:08:58,420 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577166084 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605577166084 2020-11-17 02:08:58,447 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577274627 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605577274627 2020-11-17 02:08:58,456 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577346802 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605577346802 2020-11-17 02:08:58,462 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577463406 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605577463406 2020-11-17 02:08:58,478 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577528008 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605577528008 2020-11-17 02:08:58,493 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577635845 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605577635845 2020-11-17 02:08:58,499 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577696902 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605577696902 2020-11-17 02:08:58,504 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577792609 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605577792609 2020-11-17 02:08:58,509 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577874731 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605577874731 2020-11-17 02:09:11,450 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.18 MB heapSize=128.15 MB 2020-11-17 02:09:11,484 INFO [Thread-854] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:09:12,313 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.18 MB at sequenceid=262423618 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/7bf2e5aa8d96452cb43ace2a399804c1 2020-11-17 02:09:12,325 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:09:12,356 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7bf2e5aa8d96452cb43ace2a399804c1, entries=374574, sequenceid=262423618, filesize=8.3 M 2020-11-17 02:09:12,358 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.18 MB/106093771, heapSize ~128.15 MB/134374144, currentSize=743.07 KB/760908 for e5dc5c620be19b02666a19c280ff582e in 907ms, sequenceid=262423618, compaction requested=true 2020-11-17 02:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.65 GB, freeSize=185.29 MB, max=1.83 GB, blockCount=43621, accesses=8237704, hits=7183654, hitRatio=87.20%, , cachingAccesses=7233016, cachingHits=7181946, cachingHitsRatio=99.29%, evictions=959, evicted=0, evictedPerRun=0.0 2020-11-17 02:10:36,128 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:10:38,759 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 76815 ns. 2020-11-17 02:10:38,759 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 84383 ns. 2020-11-17 02:10:45,421 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578938356 with entries=39307, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579045373 2020-11-17 02:11:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 194428 ns. 2020-11-17 02:11:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 87900 ns. 2020-11-17 02:11:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 63373 ns. 2020-11-17 02:12:22,791 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579045373 with entries=36358, filesize=128.17 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579142744 2020-11-17 02:12:28,294 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.71 MB heapSize=128.42 MB 2020-11-17 02:12:28,324 INFO [Thread-864] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:12:29,015 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.71 MB at sequenceid=262433470 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/2f227880a0dc4e09b5297acc88ad4875 2020-11-17 02:12:29,022 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:12:29,041 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2f227880a0dc4e09b5297acc88ad4875, entries=342616, sequenceid=262433470, filesize=8.2 M 2020-11-17 02:12:29,043 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.71 MB/108744384, heapSize ~128.42 MB/134661968, currentSize=667.31 KB/683325 for e5dc5c620be19b02666a19c280ff582e in 749ms, sequenceid=262433470, compaction requested=true 2020-11-17 02:12:29,045 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 02:12:29,045 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b21d8fd5c4a94e4097a86b7d9d2f93af, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7bf2e5aa8d96452cb43ace2a399804c1, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2f227880a0dc4e09b5297acc88ad4875] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=58.9 M 2020-11-17 02:12:29,073 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:12:29,074 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:12:29,103 INFO [Thread-866] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:12:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 19926 ns. 2020-11-17 02:12:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 34275 ns. 2020-11-17 02:12:41,143 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#184 average throughput is 58.54 MB/second, slept 14 time(s) and total slept time is 4695 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 02:12:41,227 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 3d4bee909a51492898fa32bf9bded666(size=60.3 M), total size for store is 48.6 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 02:12:41,227 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605579149043; duration=12sec 2020-11-17 02:12:41,381 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.74 MB heapSize=128.01 MB 2020-11-17 02:12:41,413 INFO [Thread-869] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:12:42,098 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.74 MB at sequenceid=251529313 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/0836aba35e4b45fdb310c630df3faf03 2020-11-17 02:12:42,121 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0836aba35e4b45fdb310c630df3faf03, entries=317998, sequenceid=251529313, filesize=6.8 M 2020-11-17 02:12:42,124 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.74 MB/110872950, heapSize ~128.01 MB/134223464, currentSize=176.46 KB/180696 for 82421ce8c5c8b48c573baee83caa8305 in 743ms, sequenceid=251529313, compaction requested=true 2020-11-17 02:13:00,648 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:13:16,834 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:13:33,897 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:13:40,003 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:13:58,409 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:14:11,311 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:14:35,830 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579142744 with entries=49382, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579275788 2020-11-17 02:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.65 GB, freeSize=186.02 MB, max=1.83 GB, blockCount=43606, accesses=8381294, hits=7315749, hitRatio=87.29%, , cachingAccesses=7365131, cachingHits=7314034, cachingHitsRatio=99.31%, evictions=989, evicted=0, evictedPerRun=0.0 2020-11-17 02:15:09,356 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:15:39,296 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 68022 ns. 2020-11-17 02:15:39,297 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 90590 ns. 2020-11-17 02:15:59,647 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:16:00,012 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:16:00,012 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:16:00,012 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:16:00,012 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:16:00,022 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:16:00,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:16:01,424 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579275788 with entries=31760, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579361362 2020-11-17 02:16:08,702 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:17:01,443 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.01 MB heapSize=128.00 MB 2020-11-17 02:17:01,492 INFO [Thread-884] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:17:02,686 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.01 MB at sequenceid=286728193 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/ef02f2a75941473f9348d2f23a8ab098 2020-11-17 02:17:02,695 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:17:02,715 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/ef02f2a75941473f9348d2f23a8ab098, entries=530885, sequenceid=286728193, filesize=9.1 M 2020-11-17 02:17:02,718 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.01 MB/94380293, heapSize ~128.00 MB/134219648, currentSize=76.27 KB/78105 for 5e5f05205d7e40b0141cd977dd009ba1 in 1275ms, sequenceid=286728193, compaction requested=true 2020-11-17 02:17:31,457 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.87 MB heapSize=128.15 MB 2020-11-17 02:17:31,501 INFO [Thread-888] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:17:32,337 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.87 MB at sequenceid=262448875 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/f108d156c4674bf28a2486a4fc81133e 2020-11-17 02:17:32,343 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:17:32,358 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f108d156c4674bf28a2486a4fc81133e, entries=393754, sequenceid=262448875, filesize=8.1 M 2020-11-17 02:17:32,360 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.87 MB/104723321, heapSize ~128.15 MB/134375688, currentSize=429.33 KB/439631 for e5dc5c620be19b02666a19c280ff582e in 904ms, sequenceid=262448875, compaction requested=true 2020-11-17 02:17:39,258 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 214556 ns. 2020-11-17 02:17:39,258 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18332 ns. 2020-11-17 02:17:39,258 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 26869 ns. 2020-11-17 02:17:39,258 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 93300 ns. 2020-11-17 02:17:39,258 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 57053 ns. 2020-11-17 02:17:46,640 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:17:48,052 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.31 MB heapSize=128.34 MB 2020-11-17 02:17:48,090 INFO [Thread-891] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:17:48,914 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.31 MB at sequenceid=272513610 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/46ef1ea65e73497991b31eb1bcf133bb 2020-11-17 02:17:48,954 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/46ef1ea65e73497991b31eb1bcf133bb, entries=274675, sequenceid=272513610, filesize=8.5 M 2020-11-17 02:17:48,957 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.31 MB/113571498, heapSize ~128.34 MB/134577368, currentSize=33.25 KB/34044 for fc93f6dcc564341dcc983bf30971f73d in 904ms, sequenceid=272513610, compaction requested=true 2020-11-17 02:17:48,957 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 02:17:48,958 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/85f9b5814c6c42948c03ddb5a2425b4f, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/90cd25eb9111435c9d8adf7219d3fbec, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/46ef1ea65e73497991b31eb1bcf133bb] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=222.2 M 2020-11-17 02:17:49,005 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:17:49,010 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:17:49,034 INFO [Thread-893] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:17:50,816 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:17:58,726 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:18:13,950 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579361362 with entries=49419, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579493912 2020-11-17 02:18:13,951 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605577967754 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605577967754 2020-11-17 02:18:13,961 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578041773 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605578041773 2020-11-17 02:18:13,966 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578105602 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605578105602 2020-11-17 02:18:13,972 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578191417 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605578191417 2020-11-17 02:18:13,976 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578247884 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605578247884 2020-11-17 02:18:13,981 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578342831 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605578342831 2020-11-17 02:18:13,986 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578506651 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605578506651 2020-11-17 02:18:13,991 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578596630 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605578596630 2020-11-17 02:18:13,996 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578722417 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605578722417 2020-11-17 02:18:15,617 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:18:17,953 INFO [DataStreamer for file /hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/e59bacc68ac04eb4875bd2f0bd9cae08] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:18:24,202 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:18:24,207 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:18:25,633 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:18:27,361 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:18:27,614 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:18:27,616 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:18:27,618 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:18:31,846 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:18:36,208 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#189 average throughput is 57.51 MB/second, slept 52 time(s) and total slept time is 22693 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 02:18:36,295 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:18:36,318 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into e59bacc68ac04eb4875bd2f0bd9cae08(size=219.1 M), total size for store is 33.3 G. This selection was in queue for 0sec, and took 47sec to execute. 2020-11-17 02:18:36,319 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=11, startTime=1605579468957; duration=47sec 2020-11-17 02:18:54,041 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=178.27 MB, max=1.83 GB, blockCount=43825, accesses=8453226, hits=7343712, hitRatio=86.87%, , cachingAccesses=7393237, cachingHits=7341919, cachingHitsRatio=99.31%, evictions=1019, evicted=0, evictedPerRun=0.0 2020-11-17 02:19:58,878 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579493912 with entries=38199, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579598841 2020-11-17 02:21:06,913 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.97 MB heapSize=128.00 MB 2020-11-17 02:21:06,972 INFO [Thread-905] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:07,710 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.97 MB at sequenceid=271277617 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/999a295590f4450e95baea548b1da072 2020-11-17 02:21:07,716 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:07,735 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/999a295590f4450e95baea548b1da072, entries=316608, sequenceid=271277617, filesize=8.7 M 2020-11-17 02:21:07,736 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.97 MB/110064468, heapSize ~128.00 MB/134218000, currentSize=29.70 KB/30416 for 0469ce862509756add4a8cdb00bc465c in 823ms, sequenceid=271277617, compaction requested=true 2020-11-17 02:21:07,738 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 02:21:07,738 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/def6509c09a144a2bf757c573992c900, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/e9f95dce966848a4bdb7fe58d835fecd, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/7a682d1c43d44e0daf21aaf6bce0601f, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/999a295590f4450e95baea548b1da072] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=746.7 M 2020-11-17 02:21:07,799 INFO [Thread-907] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:07,913 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:07,920 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:07,930 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:21,875 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:23,330 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:23,351 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:23,488 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:23,616 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:23,722 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:24,151 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:24,255 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:24,359 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:24,461 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:25,048 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:25,569 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:25,593 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.01 MB heapSize=128.25 MB 2020-11-17 02:21:25,620 INFO [Thread-910] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:26,479 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.01 MB at sequenceid=262460869 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/faa2bbccb3004980b3d9031a1dde7dac 2020-11-17 02:21:26,485 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:26,502 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/faa2bbccb3004980b3d9031a1dde7dac, entries=364324, sequenceid=262460869, filesize=8.3 M 2020-11-17 02:21:26,504 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.01 MB/106961854, heapSize ~128.25 MB/134481792, currentSize=866.66 KB/887457 for e5dc5c620be19b02666a19c280ff582e in 912ms, sequenceid=262460869, compaction requested=true 2020-11-17 02:21:26,505 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 02:21:26,505 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3d4bee909a51492898fa32bf9bded666, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f108d156c4674bf28a2486a4fc81133e, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/faa2bbccb3004980b3d9031a1dde7dac] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=76.7 M 2020-11-17 02:21:26,534 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:26,545 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:26,561 INFO [Thread-912] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:31,435 INFO [DataStreamer for file /hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/6d2cf5e86fec4853a82cab30302d3e79] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:21:38,739 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 89733 ns. 2020-11-17 02:21:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 87645 ns. 2020-11-17 02:21:54,743 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579598841 with entries=43048, filesize=128.33 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579714713 2020-11-17 02:21:56,780 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#193 average throughput is 30.03 MB/second, slept 18 time(s) and total slept time is 20735 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 02:21:56,840 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into d7fd546efea24815a2802ee2665700a2(size=78.2 M), total size for store is 48.6 G. This selection was in queue for 0sec, and took 30sec to execute. 2020-11-17 02:21:56,840 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605579686504; duration=30sec 2020-11-17 02:22:10,704 INFO [DataStreamer for file /hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/6d2cf5e86fec4853a82cab30302d3e79] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:22:39,583 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:22:40,490 INFO [DataStreamer for file /hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/6d2cf5e86fec4853a82cab30302d3e79] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:22:45,654 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:22:48,844 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 40535 ns. 2020-11-17 02:22:48,844 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 42339 ns. 2020-11-17 02:22:54,299 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:09,178 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:09,903 INFO [DataStreamer for file /hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/6d2cf5e86fec4853a82cab30302d3e79] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:25,501 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:25,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:26,493 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:26,555 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:27,034 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:27,331 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:27,333 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:27,358 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:27,586 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:27,959 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:30,945 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:31,126 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:31,128 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:31,346 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:34,499 INFO [DataStreamer for file /hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/6d2cf5e86fec4853a82cab30302d3e79] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:35,710 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:35,712 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:35,714 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:36,261 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:36,376 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:36,378 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:36,522 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:36,524 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:36,527 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:36,645 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:36,784 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:36,966 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:37,218 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:37,221 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:37,360 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:37,432 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:37,950 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:37,985 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:38,214 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:38,242 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 76244 ns. 2020-11-17 02:23:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 214997 ns. 2020-11-17 02:23:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 90468 ns. 2020-11-17 02:23:39,229 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:40,400 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:40,404 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:41,359 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:41,461 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:41,964 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:44,977 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:48,133 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:50,870 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#191 average throughput is 51.25 MB/second, slept 148 time(s) and total slept time is 76856 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 02:23:50,960 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 6d2cf5e86fec4853a82cab30302d3e79(size=729.1 M), total size for store is 42.6 G. This selection was in queue for 0sec, and took 2mins, 43sec to execute. 2020-11-17 02:23:50,960 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=11, startTime=1605579667737; duration=2mins, 43sec 2020-11-17 02:23:52,627 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.48 MB heapSize=128.04 MB 2020-11-17 02:23:52,675 INFO [Thread-928] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:53,951 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.48 MB at sequenceid=260959069 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/20b9e4682dc5406daa309703117e8b16 2020-11-17 02:23:53,975 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/20b9e4682dc5406daa309703117e8b16, entries=576427, sequenceid=260959069, filesize=8.9 M 2020-11-17 02:23:53,977 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.48 MB/89635178, heapSize ~128.04 MB/134255496, currentSize=209.69 KB/214718 for 73aae5715542e1a5370f7540d2117abc in 1349ms, sequenceid=260959069, compaction requested=true 2020-11-17 02:23:53,978 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 02:23:53,978 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/3883ef3c9ab74dd79f03904c75341083, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/9897a5cb004942a6a15d7a33e9d4ef39, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/20b9e4682dc5406daa309703117e8b16] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=45.7 M 2020-11-17 02:23:54,053 INFO [Thread-930] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:54,053 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:23:54,418 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:24:01,225 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#195 average throughput is 58.44 MB/second, slept 5 time(s) and total slept time is 616 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 02:24:01,416 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 6485241470c94f1fb50f434764b284e3(size=46.2 M), total size for store is 12.4 G. This selection was in queue for 0sec, and took 7sec to execute. 2020-11-17 02:24:01,419 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=10, startTime=1605579833977; duration=7sec 2020-11-17 02:24:05,245 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:24:16,779 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:24:19,205 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579714713 with entries=53895, filesize=128.71 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579859144 2020-11-17 02:24:22,439 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:24:26,969 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:24:30,019 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=169.68 MB, max=1.83 GB, blockCount=44049, accesses=9112656, hits=7846174, hitRatio=86.10%, , cachingAccesses=7895526, cachingHits=7843851, cachingHitsRatio=99.35%, evictions=1049, evicted=0, evictedPerRun=0.0 2020-11-17 02:24:38,771 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:24:46,849 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:24:47,137 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=98.57 MB heapSize=128.00 MB 2020-11-17 02:24:47,183 INFO [Thread-934] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:24:48,324 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.57 MB at sequenceid=259118547 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/84cc325adf9444ac984222a0079388a0 2020-11-17 02:24:48,405 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/84cc325adf9444ac984222a0079388a0, entries=405631, sequenceid=259118547, filesize=7.9 M 2020-11-17 02:24:48,408 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.57 MB/103356441, heapSize ~128.00 MB/134219328, currentSize=200.50 KB/205316 for 06f7ca9534710e37d90d9f8345864259 in 1272ms, sequenceid=259118547, compaction requested=true 2020-11-17 02:24:48,409 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 02:24:48,410 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/f04d72b888f847598c29ff6a0a3d1dba, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/bb69da67310341f6a6c28f260eac1acc, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/84cc325adf9444ac984222a0079388a0] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=21.8 M 2020-11-17 02:24:48,521 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:24:48,547 INFO [Thread-936] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:24:48,779 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:24:52,397 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:24:53,545 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#197 average throughput is 59.51 MB/second, slept 3 time(s) and total slept time is 818 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 02:24:53,591 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:24:53,615 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 960adb996e4c440eb5aa5d687ec07112(size=21.9 M), total size for store is 55.3 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 02:24:53,615 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605579888408; duration=5sec 2020-11-17 02:24:57,605 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:25:02,540 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.25 MB heapSize=128.08 MB 2020-11-17 02:25:02,586 INFO [Thread-938] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:25:03,384 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.25 MB at sequenceid=262471873 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/6d2bd30ec9df4462b2c54ffcb9bd5a17 2020-11-17 02:25:03,392 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:25:03,423 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6d2bd30ec9df4462b2c54ffcb9bd5a17, entries=358366, sequenceid=262471873, filesize=8.2 M 2020-11-17 02:25:03,430 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.25 MB/107217003, heapSize ~128.08 MB/134297648, currentSize=918.26 KB/940296 for e5dc5c620be19b02666a19c280ff582e in 890ms, sequenceid=262471873, compaction requested=true 2020-11-17 02:25:03,472 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.28 MB heapSize=128.06 MB 2020-11-17 02:25:03,502 INFO [Thread-940] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:25:04,583 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.28 MB at sequenceid=251551295 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/5fc3d01f322b41f3a81d1337def3acdb 2020-11-17 02:25:04,609 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/5fc3d01f322b41f3a81d1337def3acdb, entries=324636, sequenceid=251551295, filesize=6.9 M 2020-11-17 02:25:04,611 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.28 MB/110394396, heapSize ~128.06 MB/134280432, currentSize=148.67 KB/152235 for 82421ce8c5c8b48c573baee83caa8305 in 1140ms, sequenceid=251551295, compaction requested=true 2020-11-17 02:25:04,612 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 02:25:04,612 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/eb28451b27844712a385ceef797e951e, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0836aba35e4b45fdb310c630df3faf03, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/5fc3d01f322b41f3a81d1337def3acdb] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=70.9 M 2020-11-17 02:25:04,675 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:25:04,679 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:25:04,711 INFO [Thread-942] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:25:05,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:25:11,098 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:25:23,966 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:25:24,901 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#200 average throughput is 57.76 MB/second, slept 22 time(s) and total slept time is 7121 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 02:25:24,992 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into d1a83fd2619746b198acb4fa3896fb8a(size=70.9 M), total size for store is 11.0 G. This selection was in queue for 0sec, and took 20sec to execute. 2020-11-17 02:25:24,992 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605579904612; duration=20sec 2020-11-17 02:25:29,065 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:25:40,038 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:25:46,884 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579859144 with entries=32176, filesize=128.14 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579946855 2020-11-17 02:25:47,570 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:25:56,566 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:26:01,123 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:26:05,133 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:26:10,763 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:26:20,629 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:26:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 90351 ns. 2020-11-17 02:26:47,985 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:27:25,152 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579946855 with entries=36790, filesize=128.26 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580045108 2020-11-17 02:27:30,857 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:27:38,871 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 89849 ns. 2020-11-17 02:27:49,523 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 45883 ns. 2020-11-17 02:27:49,523 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18578 ns. 2020-11-17 02:27:56,264 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:28:04,960 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:29:17,649 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580045108 with entries=41742, filesize=128.08 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580157596 2020-11-17 02:29:23,943 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:29:35,446 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.65 GB, freeSize=191.06 MB, max=1.83 GB, blockCount=43531, accesses=9563407, hits=8273083, hitRatio=86.51%, , cachingAccesses=8322486, cachingHits=8270730, cachingHitsRatio=99.38%, evictions=1079, evicted=0, evictedPerRun=0.0 2020-11-17 02:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 02:29:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 219304 ns. 2020-11-17 02:29:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 78068 ns. 2020-11-17 02:29:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 107635 ns. 2020-11-17 02:29:48,788 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.23 MB heapSize=128.10 MB 2020-11-17 02:29:48,829 INFO [Thread-957] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:29:49,657 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.23 MB at sequenceid=262486407 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/874cb73dd53e4c2994209c4b4597329f 2020-11-17 02:29:49,666 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:29:49,696 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/874cb73dd53e4c2994209c4b4597329f, entries=415695, sequenceid=262486407, filesize=7.9 M 2020-11-17 02:29:49,698 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~98.23 MB/103006487, heapSize ~128.10 MB/134324152, currentSize=510.65 KB/522905 for e5dc5c620be19b02666a19c280ff582e in 910ms, sequenceid=262486407, compaction requested=true 2020-11-17 02:29:49,699 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 02:29:49,699 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d7fd546efea24815a2802ee2665700a2, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6d2bd30ec9df4462b2c54ffcb9bd5a17, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/874cb73dd53e4c2994209c4b4597329f] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=94.3 M 2020-11-17 02:29:49,734 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:29:49,748 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:29:49,802 INFO [Thread-959] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:30:09,059 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#202 average throughput is 57.43 MB/second, slept 20 time(s) and total slept time is 7453 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 02:30:09,158 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 470f6559cb3241b3a7bde43b58696be5(size=95.8 M), total size for store is 48.6 G. This selection was in queue for 0sec, and took 19sec to execute. 2020-11-17 02:30:09,159 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605580189698; duration=19sec 2020-11-17 02:30:27,966 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580157596 with entries=26897, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580227934 2020-11-17 02:31:03,280 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.06 MB heapSize=128.08 MB 2020-11-17 02:31:03,329 INFO [Thread-965] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:31:03,951 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.06 MB at sequenceid=251562099 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/b54f41f61e334b9a941f931e74603b56 2020-11-17 02:31:03,957 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:31:03,977 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b54f41f61e334b9a941f931e74603b56, entries=314575, sequenceid=251562099, filesize=6.4 M 2020-11-17 02:31:03,979 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.06 MB/111216099, heapSize ~128.08 MB/134305112, currentSize=89.22 KB/91359 for 82421ce8c5c8b48c573baee83caa8305 in 699ms, sequenceid=251562099, compaction requested=true 2020-11-17 02:31:10,886 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:31:39,576 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 83179 ns. 2020-11-17 02:31:57,786 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580227934 with entries=32661, filesize=128.17 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580317764 2020-11-17 02:32:58,725 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 20709 ns. 2020-11-17 02:32:58,725 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 36808 ns. 2020-11-17 02:33:03,252 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.85 MB heapSize=128.08 MB 2020-11-17 02:33:03,294 INFO [Thread-974] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:33:03,991 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.85 MB at sequenceid=272541593 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/e0ee001b7f4e4061b13ade0bf33e7ad9 2020-11-17 02:33:03,998 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:33:04,016 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/e0ee001b7f4e4061b13ade0bf33e7ad9, entries=277451, sequenceid=272541593, filesize=8.5 M 2020-11-17 02:33:04,017 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~107.85 MB/113094040, heapSize ~128.08 MB/134299048, currentSize=15.98 KB/16360 for fc93f6dcc564341dcc983bf30971f73d in 765ms, sequenceid=272541593, compaction requested=true 2020-11-17 02:33:18,053 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:33:20,159 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580317764 with entries=31566, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580400132 2020-11-17 02:33:28,346 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.50 MB heapSize=128.34 MB 2020-11-17 02:33:28,384 INFO [Thread-977] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:33:29,094 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.50 MB at sequenceid=262497664 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/7cf59b17ee85410a8e1cdfb5849995fe 2020-11-17 02:33:29,103 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:33:29,120 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7cf59b17ee85410a8e1cdfb5849995fe, entries=357984, sequenceid=262497664, filesize=8.2 M 2020-11-17 02:33:29,121 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.50 MB/107481851, heapSize ~128.34 MB/134576048, currentSize=382.48 KB/391661 for e5dc5c620be19b02666a19c280ff582e in 775ms, sequenceid=262497664, compaction requested=true 2020-11-17 02:33:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 63308 ns. 2020-11-17 02:34:33,532 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.90 MB heapSize=128.06 MB 2020-11-17 02:34:33,592 INFO [Thread-982] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:34:34,338 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.90 MB at sequenceid=271317306 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/ee21a9844ef94eff933fdb727150cafc 2020-11-17 02:34:34,345 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:34:34,375 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/ee21a9844ef94eff933fdb727150cafc, entries=318379, sequenceid=271317306, filesize=8.6 M 2020-11-17 02:34:34,376 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.90 MB/109998018, heapSize ~128.06 MB/134284144, currentSize=58.85 KB/60261 for 0469ce862509756add4a8cdb00bc465c in 844ms, sequenceid=271317306, compaction requested=true 2020-11-17 02:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.65 GB, freeSize=191.66 MB, max=1.83 GB, blockCount=43521, accesses=9583703, hits=8275372, hitRatio=86.35%, , cachingAccesses=8324768, cachingHits=8273007, cachingHitsRatio=99.38%, evictions=1109, evicted=0, evictedPerRun=0.0 2020-11-17 02:34:55,874 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:35:21,070 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580400132 with entries=44717, filesize=128.17 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580521030 2020-11-17 02:35:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 90667 ns. 2020-11-17 02:35:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 53484 ns. 2020-11-17 02:35:38,750 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 188747 ns. 2020-11-17 02:36:31,040 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:36:31,045 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:36:33,798 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:36:36,209 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:37:02,605 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580521030 with entries=43371, filesize=128.16 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580622558 2020-11-17 02:37:19,962 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.63 MB heapSize=128.17 MB 2020-11-17 02:37:20,018 INFO [Thread-992] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:37:20,835 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.63 MB at sequenceid=259158987 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/ccf84e9341d14528807021b1a0911ac9 2020-11-17 02:37:20,842 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:37:20,861 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/ccf84e9341d14528807021b1a0911ac9, entries=381040, sequenceid=259158987, filesize=7.3 M 2020-11-17 02:37:20,862 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.63 MB/105518809, heapSize ~128.17 MB/134391624, currentSize=66.79 KB/68391 for 06f7ca9534710e37d90d9f8345864259 in 900ms, sequenceid=259158987, compaction requested=true 2020-11-17 02:37:35,523 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:37:39,340 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 76715 ns. 2020-11-17 02:38:08,724 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 34690 ns. 2020-11-17 02:38:08,724 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14117 ns. 2020-11-17 02:38:14,085 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.70 MB heapSize=128.14 MB 2020-11-17 02:38:14,133 INFO [Thread-996] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:38:14,940 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.70 MB at sequenceid=262513149 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/87d2368c8fcd4019869beedf079383a7 2020-11-17 02:38:14,946 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:38:14,975 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/87d2368c8fcd4019869beedf079383a7, entries=352683, sequenceid=262513149, filesize=8.2 M 2020-11-17 02:38:14,977 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.70 MB/107690081, heapSize ~128.14 MB/134369400, currentSize=45.65 KB/46745 for e5dc5c620be19b02666a19c280ff582e in 892ms, sequenceid=262513149, compaction requested=true 2020-11-17 02:38:14,978 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 02:38:14,978 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/470f6559cb3241b3a7bde43b58696be5, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7cf59b17ee85410a8e1cdfb5849995fe, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/87d2368c8fcd4019869beedf079383a7] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=112.2 M 2020-11-17 02:38:15,007 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:38:15,010 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:38:15,035 INFO [Thread-998] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:38:39,152 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 67707 ns. 2020-11-17 02:38:39,361 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#209 average throughput is 53.96 MB/second, slept 23 time(s) and total slept time is 9354 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 02:38:39,454 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 59a81f420aef43d7a34781eff8f52ac3(size=113.7 M), total size for store is 48.7 G. This selection was in queue for 0sec, and took 24sec to execute. 2020-11-17 02:38:39,454 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605580694977; duration=24sec 2020-11-17 02:39:03,137 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:39:10,912 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:39:28,050 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:39:30,080 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580622558 with entries=53988, filesize=128.29 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580770052 2020-11-17 02:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=174.80 MB, max=1.83 GB, blockCount=44114, accesses=9894799, hits=8563621, hitRatio=86.55%, , cachingAccesses=8614521, cachingHits=8561252, cachingHitsRatio=99.38%, evictions=1139, evicted=0, evictedPerRun=0.0 2020-11-17 02:40:21,383 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.88 MB heapSize=128.00 MB 2020-11-17 02:40:21,435 INFO [Thread-1005] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:22,425 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:22,804 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.88 MB at sequenceid=261011527 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/f88a1951a1554b01bd33ddf9436933f1 2020-11-17 02:40:22,831 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/f88a1951a1554b01bd33ddf9436933f1, entries=576637, sequenceid=261011527, filesize=9.0 M 2020-11-17 02:40:22,837 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.88 MB/90052161, heapSize ~128.00 MB/134222920, currentSize=77.76 KB/79624 for 73aae5715542e1a5370f7540d2117abc in 1453ms, sequenceid=261011527, compaction requested=true 2020-11-17 02:40:22,839 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 02:40:22,839 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/30de5b3f653c4ecda7696c4ae4a1b340, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/0e49be7810c748eebaaa9369754cd54d, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/6485241470c94f1fb50f434764b284e3, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/f88a1951a1554b01bd33ddf9436933f1] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=397.7 M 2020-11-17 02:40:22,911 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:22,925 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:22,935 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:23,057 INFO [Thread-1007] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:23,838 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:23,846 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:32,479 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:32,485 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:32,881 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:43,211 INFO [DataStreamer for file /hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/1b4769138044494cb0e1b79523dd0948] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:46,406 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:50,264 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:55,052 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:55,351 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:56,510 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:58,683 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:40:58,686 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:01,987 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:01,988 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:02,125 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:02,337 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.56 MB heapSize=128.00 MB 2020-11-17 02:41:02,381 INFO [Thread-1012] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:02,918 INFO [DataStreamer for file /hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/1b4769138044494cb0e1b79523dd0948] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:03,432 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.56 MB at sequenceid=286807447 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/270228eaa23e4bdb95c476f9aea83b4a 2020-11-17 02:41:03,445 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:03,466 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/270228eaa23e4bdb95c476f9aea83b4a, entries=522231, sequenceid=286807447, filesize=9.1 M 2020-11-17 02:41:03,468 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.56 MB/94963322, heapSize ~128.00 MB/134217816, currentSize=44.13 KB/45184 for 5e5f05205d7e40b0141cd977dd009ba1 in 1132ms, sequenceid=286807447, compaction requested=true 2020-11-17 02:41:04,596 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:05,931 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:07,017 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:07,019 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:07,187 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:07,189 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:08,051 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:08,054 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:08,056 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:08,058 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:08,060 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:08,061 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:08,192 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:08,310 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:08,788 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:08,913 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:08,915 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:09,029 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:09,156 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:09,663 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:09,867 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:09,988 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:10,527 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:10,720 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:10,729 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:11,447 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:11,451 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:11,524 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:16,815 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:16,817 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:16,820 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:16,823 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:17,824 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:17,950 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:26,224 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.69 MB heapSize=128.00 MB 2020-11-17 02:41:26,255 INFO [Thread-1016] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:27,367 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.69 MB at sequenceid=290702742 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/51ca25e4025c45349cfca590053b3374 2020-11-17 02:41:27,373 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:27,391 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/51ca25e4025c45349cfca590053b3374, entries=546041, sequenceid=290702742, filesize=9.1 M 2020-11-17 02:41:27,393 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~88.69 MB/92993741, heapSize ~128.00 MB/134217832, currentSize=36.67 KB/37553 for bdaa31a63b7f3d06d9909b7560333e79 in 1169ms, sequenceid=290702742, compaction requested=true 2020-11-17 02:41:27,394 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-17 02:41:27,394 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/3c4cbc976ee74d0798e8aac9f64d31a7, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/9ac91671065c4ab49adfdf584f83f42e, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/51ca25e4025c45349cfca590053b3374] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp, totalSize=84.6 M 2020-11-17 02:41:27,437 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:27,457 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:27,465 INFO [Thread-1018] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:28,423 INFO [DataStreamer for file /hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/1b4769138044494cb0e1b79523dd0948] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:28,448 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:28,600 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#211 average throughput is 53.16 MB/second, slept 48 time(s) and total slept time is 13047 ms. 1 active operations remaining, total limit is 57.69 MB/second 2020-11-17 02:41:28,671 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 1b4769138044494cb0e1b79523dd0948(size=386.2 M), total size for store is 12.4 G. This selection was in queue for 0sec, and took 1mins, 5sec to execute. 2020-11-17 02:41:28,671 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605580822837; duration=1mins, 5sec 2020-11-17 02:41:36,770 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:37,636 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 59898 ns. 2020-11-17 02:41:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 181228 ns. 2020-11-17 02:41:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 77111 ns. 2020-11-17 02:41:39,326 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:40,837 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:41:43,812 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: bdaa31a63b7f3d06d9909b7560333e79#e#compaction#214 average throughput is 50.37 MB/second, slept 13 time(s) and total slept time is 3563 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 02:41:43,901 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of bdaa31a63b7f3d06d9909b7560333e79 into 0d76d24b3bab47f2aa9d21f50a4ba84d(size=83.7 M), total size for store is 31.8 G. This selection was in queue for 0sec, and took 16sec to execute. 2020-11-17 02:41:43,901 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., storeName=e, priority=11, startTime=1605580887393; duration=16sec 2020-11-17 02:41:56,666 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580770052 with entries=52463, filesize=128.15 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580916608 2020-11-17 02:41:56,667 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578805568 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605578805568 2020-11-17 02:41:56,723 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578938356 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605578938356 2020-11-17 02:41:56,734 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579045373 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605579045373 2020-11-17 02:41:56,743 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579142744 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605579142744 2020-11-17 02:41:56,751 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579275788 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605579275788 2020-11-17 02:41:56,757 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579361362 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605579361362 2020-11-17 02:41:56,763 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579493912 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605579493912 2020-11-17 02:41:56,768 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579598841 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605579598841 2020-11-17 02:41:56,779 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579714713 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605579714713 2020-11-17 02:41:56,786 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579859144 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605579859144 2020-11-17 02:41:56,791 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605579946855 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605579946855 2020-11-17 02:41:56,797 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580045108 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605580045108 2020-11-17 02:41:56,803 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580157596 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605580157596 2020-11-17 02:42:07,174 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.13 MB heapSize=128.02 MB 2020-11-17 02:42:07,210 INFO [Thread-1024] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:42:08,092 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.13 MB at sequenceid=262524622 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/3ca6fdf31e674a0383ecba0fc2430eff 2020-11-17 02:42:08,099 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:42:08,120 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3ca6fdf31e674a0383ecba0fc2430eff, entries=416470, sequenceid=262524622, filesize=8.0 M 2020-11-17 02:42:08,124 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.13 MB/102897117, heapSize ~128.02 MB/134242952, currentSize=286.61 KB/293490 for e5dc5c620be19b02666a19c280ff582e in 950ms, sequenceid=262524622, compaction requested=true 2020-11-17 02:42:31,935 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:43:01,144 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:43:08,858 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 9186 ns. 2020-11-17 02:43:08,858 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 43152 ns. 2020-11-17 02:43:18,049 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:43:38,838 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 94293 ns. 2020-11-17 02:44:00,357 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580916608 with entries=45250, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581040333 2020-11-17 02:44:34,278 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=104.74 MB heapSize=128.00 MB 2020-11-17 02:44:34,337 INFO [Thread-1033] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:44:35,291 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.74 MB at sequenceid=251585575 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/840a43774190470aa8baf4360f9ce9d4 2020-11-17 02:44:35,298 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:44:35,319 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/840a43774190470aa8baf4360f9ce9d4, entries=331213, sequenceid=251585575, filesize=6.8 M 2020-11-17 02:44:35,321 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.74 MB/109828396, heapSize ~128.00 MB/134220432, currentSize=25.00 KB/25596 for 82421ce8c5c8b48c573baee83caa8305 in 1043ms, sequenceid=251585575, compaction requested=true 2020-11-17 02:44:35,323 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 02:44:35,323 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d1a83fd2619746b198acb4fa3896fb8a, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b54f41f61e334b9a941f931e74603b56, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/840a43774190470aa8baf4360f9ce9d4] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=84.1 M 2020-11-17 02:44:35,351 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:44:35,355 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:44:35,379 INFO [Thread-1035] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.65 GB, freeSize=186.30 MB, max=1.83 GB, blockCount=43755, accesses=10177015, hits=8774294, hitRatio=86.22%, , cachingAccesses=8824862, cachingHits=8771537, cachingHitsRatio=99.40%, evictions=1169, evicted=0, evictedPerRun=0.0 2020-11-17 02:44:39,011 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 78656 ns. 2020-11-17 02:44:56,027 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:44:56,599 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:44:59,162 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#217 average throughput is 57.84 MB/second, slept 27 time(s) and total slept time is 10851 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 02:44:59,233 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 68be103afc1f48d9a0bf52180b74c093(size=83.9 M), total size for store is 11.0 G. This selection was in queue for 0sec, and took 23sec to execute. 2020-11-17 02:44:59,234 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605581075322; duration=23sec 2020-11-17 02:45:11,881 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:45:26,549 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:45:41,014 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=104.00 MB heapSize=128.16 MB 2020-11-17 02:45:41,050 INFO [Thread-1040] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:45:41,780 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.00 MB at sequenceid=262535477 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/551f0a9d2e1c4060a88026c28b215ff1 2020-11-17 02:45:41,790 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:45:41,821 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/551f0a9d2e1c4060a88026c28b215ff1, entries=334750, sequenceid=262535477, filesize=8.2 M 2020-11-17 02:45:41,824 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.00 MB/109050719, heapSize ~128.16 MB/134381080, currentSize=646.61 KB/662125 for e5dc5c620be19b02666a19c280ff582e in 811ms, sequenceid=262535477, compaction requested=true 2020-11-17 02:45:53,732 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581040333 with entries=42665, filesize=128.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581153689 2020-11-17 02:45:53,732 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580227934 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605580227934 2020-11-17 02:45:59,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:46:06,222 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:46:20,250 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:46:42,808 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:47:05,452 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:47:25,747 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:47:38,864 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 219233 ns. 2020-11-17 02:47:38,865 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 44194 ns. 2020-11-17 02:47:38,865 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 56038 ns. 2020-11-17 02:47:41,880 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:47:49,733 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:48:06,472 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.96 MB heapSize=128.00 MB 2020-11-17 02:48:06,505 INFO [Thread-1049] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:48:07,428 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.96 MB at sequenceid=271357993 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/51a8a0ad5aa4457e87ae4361595cae35 2020-11-17 02:48:07,434 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:48:07,452 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/51a8a0ad5aa4457e87ae4361595cae35, entries=316012, sequenceid=271357993, filesize=8.7 M 2020-11-17 02:48:07,454 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.96 MB/110061007, heapSize ~128.00 MB/134218152, currentSize=33.57 KB/34375 for 0469ce862509756add4a8cdb00bc465c in 983ms, sequenceid=271357993, compaction requested=true 2020-11-17 02:48:08,966 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 16073 ns. 2020-11-17 02:48:08,966 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 45884 ns. 2020-11-17 02:48:20,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:48:24,943 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581153689 with entries=56074, filesize=128.23 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581304916 2020-11-17 02:48:39,540 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 55549 ns. 2020-11-17 02:48:54,019 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.06 MB heapSize=128.08 MB 2020-11-17 02:48:54,066 INFO [Thread-1053] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:48:54,806 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.06 MB at sequenceid=272570278 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/acd20cc678f54642adef84b4217812b4 2020-11-17 02:48:54,812 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:48:54,828 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/acd20cc678f54642adef84b4217812b4, entries=274460, sequenceid=272570278, filesize=8.5 M 2020-11-17 02:48:54,830 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.06 MB/113313786, heapSize ~128.08 MB/134300008, currentSize=619.58 KB/634445 for fc93f6dcc564341dcc983bf30971f73d in 812ms, sequenceid=272570278, compaction requested=true 2020-11-17 02:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.65 GB, freeSize=185.34 MB, max=1.83 GB, blockCount=43775, accesses=10203901, hits=8779549, hitRatio=86.04%, , cachingAccesses=8830140, cachingHits=8776788, cachingHitsRatio=99.40%, evictions=1199, evicted=0, evictedPerRun=0.0 2020-11-17 02:50:24,564 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.85 MB heapSize=128.30 MB 2020-11-17 02:50:24,599 INFO [Thread-1058] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:50:25,354 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.85 MB at sequenceid=262549672 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/3c7c64175f2e4a9b9051f6f30e8b9594 2020-11-17 02:50:25,360 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:50:25,377 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3c7c64175f2e4a9b9051f6f30e8b9594, entries=367757, sequenceid=262549672, filesize=8.2 M 2020-11-17 02:50:25,379 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.85 MB/106798203, heapSize ~128.30 MB/134529864, currentSize=814.58 KB/834128 for e5dc5c620be19b02666a19c280ff582e in 815ms, sequenceid=262549672, compaction requested=true 2020-11-17 02:50:25,380 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 02:50:25,380 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3ca6fdf31e674a0383ecba0fc2430eff, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/551f0a9d2e1c4060a88026c28b215ff1, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3c7c64175f2e4a9b9051f6f30e8b9594] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.4 M 2020-11-17 02:50:25,409 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:50:25,411 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:50:25,442 INFO [Thread-1060] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:50:30,334 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#222 average throughput is 61.88 MB/second, slept 6 time(s) and total slept time is 1686 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 02:50:30,402 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 6a01d43e80e248dea2aaa9bb8f8cc1f7(size=25.5 M), total size for store is 48.7 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 02:50:30,402 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605581425379; duration=5sec 2020-11-17 02:50:38,753 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 50110 ns. 2020-11-17 02:50:40,682 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:50:41,997 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581304916 with entries=50090, filesize=128.13 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581441959 2020-11-17 02:50:41,997 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580317764 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605580317764 2020-11-17 02:50:42,012 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580400132 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605580400132 2020-11-17 02:50:42,019 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580521030 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605580521030 2020-11-17 02:52:28,949 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581441959 with entries=39358, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581548915 2020-11-17 02:53:09,249 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 13252 ns. 2020-11-17 02:53:09,249 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 34047 ns. 2020-11-17 02:53:39,584 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 91755 ns. 2020-11-17 02:53:39,584 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 41491 ns. 2020-11-17 02:53:39,584 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 226037 ns. 2020-11-17 02:53:39,585 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 54341 ns. 2020-11-17 02:54:11,390 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.53 MB heapSize=128.15 MB 2020-11-17 02:54:11,437 INFO [Thread-1073] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:54:12,253 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.53 MB at sequenceid=262560973 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/d797defcac52422a829a344c1debf0e0 2020-11-17 02:54:12,260 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:54:12,280 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d797defcac52422a829a344c1debf0e0, entries=412397, sequenceid=262560973, filesize=8.1 M 2020-11-17 02:54:12,289 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.53 MB/103311549, heapSize ~128.15 MB/134371560, currentSize=458.83 KB/469837 for e5dc5c620be19b02666a19c280ff582e in 900ms, sequenceid=262560973, compaction requested=true 2020-11-17 02:54:25,906 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:54:29,888 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581548915 with entries=44333, filesize=128.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581669859 2020-11-17 02:54:33,010 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.65 GB, freeSize=185.22 MB, max=1.83 GB, blockCount=43777, accesses=10212629, hits=8783333, hitRatio=86.00%, , cachingAccesses=8833929, cachingHits=8780571, cachingHitsRatio=99.40%, evictions=1229, evicted=0, evictedPerRun=0.0 2020-11-17 02:54:39,449 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:55:00,612 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.42 MB heapSize=128.50 MB 2020-11-17 02:55:00,670 INFO [Thread-1077] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:55:01,802 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.42 MB at sequenceid=261057037 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/f814e36eeea54109bf5e13b3e19c53ae 2020-11-17 02:55:01,814 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:55:01,847 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/f814e36eeea54109bf5e13b3e19c53ae, entries=577674, sequenceid=261057037, filesize=8.9 M 2020-11-17 02:55:01,853 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.42 MB/89571111, heapSize ~128.50 MB/134744424, currentSize=161.91 KB/165793 for 73aae5715542e1a5370f7540d2117abc in 1241ms, sequenceid=261057037, compaction requested=true 2020-11-17 02:55:08,557 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:55:39,675 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 56316 ns. 2020-11-17 02:56:12,796 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581669859 with entries=37582, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581772752 2020-11-17 02:56:34,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:56:35,346 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:56:35,347 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:57:06,257 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:57:13,807 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:57:18,273 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:57:26,078 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:57:33,348 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.91 MB heapSize=128.73 MB 2020-11-17 02:57:33,388 INFO [Thread-1086] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:57:34,235 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.92 MB at sequenceid=262571014 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/e4fb0be76cec4c81bcec5eceba52d310 2020-11-17 02:57:34,281 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e4fb0be76cec4c81bcec5eceba52d310, entries=343844, sequenceid=262571014, filesize=8.1 M 2020-11-17 02:57:34,284 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.92 MB/108963277, heapSize ~128.73 MB/134985968, currentSize=1.21 MB/1272959 for e5dc5c620be19b02666a19c280ff582e in 936ms, sequenceid=262571014, compaction requested=true 2020-11-17 02:57:34,285 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 02:57:34,285 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6a01d43e80e248dea2aaa9bb8f8cc1f7, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d797defcac52422a829a344c1debf0e0, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e4fb0be76cec4c81bcec5eceba52d310] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.7 M 2020-11-17 02:57:34,329 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:57:34,348 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:57:34,380 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:57:34,381 INFO [Thread-1088] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:57:36,104 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581772752 with entries=29759, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581856079 2020-11-17 02:57:42,345 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#226 average throughput is 63.43 MB/second, slept 9 time(s) and total slept time is 2165 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 02:57:42,433 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 20cd1c26ee9a4f9bb12699cda2dbf9b6(size=43.1 M), total size for store is 48.7 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 02:57:42,433 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605581854284; duration=8sec 2020-11-17 02:58:09,652 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 12843 ns. 2020-11-17 02:58:09,653 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 49637 ns. 2020-11-17 02:58:12,551 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:58:22,489 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:58:29,100 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=97.31 MB heapSize=128.01 MB 2020-11-17 02:58:29,140 INFO [Thread-1092] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:58:30,039 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.31 MB at sequenceid=259224381 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/7a468eb1518c40f1b82276714e70ecd5 2020-11-17 02:58:30,045 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:58:30,066 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/7a468eb1518c40f1b82276714e70ecd5, entries=419575, sequenceid=259224381, filesize=8.2 M 2020-11-17 02:58:30,068 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~97.31 MB/102041036, heapSize ~128.01 MB/134224088, currentSize=162.80 KB/166711 for 06f7ca9534710e37d90d9f8345864259 in 968ms, sequenceid=259224381, compaction requested=true 2020-11-17 02:58:30,068 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 02:58:30,069 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/960adb996e4c440eb5aa5d687ec07112, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/ccf84e9341d14528807021b1a0911ac9, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/7a468eb1518c40f1b82276714e70ecd5] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=37.3 M 2020-11-17 02:58:30,096 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:58:30,108 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:58:30,149 INFO [Thread-1094] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:58:31,034 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:58:36,584 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:58:37,479 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.23 MB heapSize=128.35 MB 2020-11-17 02:58:37,506 INFO [Thread-1096] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:58:38,531 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.23 MB at sequenceid=251609811 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/a8ca16932f9d41bf949659a51155accc 2020-11-17 02:58:38,564 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a8ca16932f9d41bf949659a51155accc, entries=329469, sequenceid=251609811, filesize=6.8 M 2020-11-17 02:58:38,566 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.23 MB/110341510, heapSize ~128.35 MB/134588208, currentSize=723.10 KB/740452 for 82421ce8c5c8b48c573baee83caa8305 in 1087ms, sequenceid=251609811, compaction requested=true 2020-11-17 02:58:38,654 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#228 average throughput is 58.13 MB/second, slept 7 time(s) and total slept time is 2457 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 02:58:38,720 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into eae24701c9bb449fb4f6477235fc6c6a(size=37.7 M), total size for store is 55.3 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 02:58:38,720 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605581910068; duration=8sec 2020-11-17 02:58:53,242 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:59:05,533 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:59:12,060 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:59:19,230 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:59:23,478 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:59:25,402 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581856079 with entries=39894, filesize=128.08 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581965372 2020-11-17 02:59:25,403 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580622558 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605580622558 2020-11-17 02:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=174.83 MB, max=1.83 GB, blockCount=44149, accesses=11097782, hits=9651798, hitRatio=86.97%, , cachingAccesses=9702783, cachingHits=9649014, cachingHitsRatio=99.45%, evictions=1259, evicted=0, evictedPerRun=0.0 2020-11-17 02:59:38,753 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 215788 ns. 2020-11-17 02:59:38,753 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 47587 ns. 2020-11-17 02:59:38,754 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 98222 ns. 2020-11-17 02:59:38,754 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 104667 ns. 2020-11-17 02:59:43,353 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 02:59:57,426 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:00:09,848 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:00:39,282 INFO [regionserver/cvp21:16201.Chore.1] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of __test-table_cvp23__,,1598623751841.bac2756ecc315bfd457fb71d81dd3eb5. because __status_family___ has an old edit so flush to free WALs after random delay 223167 ms 2020-11-17 03:00:49,246 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581965372 with entries=30980, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582049202 2020-11-17 03:01:39,384 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 53762 ns. 2020-11-17 03:01:43,370 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.07 MB heapSize=128.10 MB 2020-11-17 03:01:43,407 INFO [Thread-1107] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:01:44,181 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.07 MB at sequenceid=271397490 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/80460a63a95945e39d28e29a10d79b3c 2020-11-17 03:01:44,190 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:01:44,207 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/80460a63a95945e39d28e29a10d79b3c, entries=316507, sequenceid=271397490, filesize=8.7 M 2020-11-17 03:01:44,209 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.07 MB/110174884, heapSize ~128.10 MB/134319920, currentSize=23.22 KB/23780 for 0469ce862509756add4a8cdb00bc465c in 839ms, sequenceid=271397490, compaction requested=true 2020-11-17 03:01:44,210 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 03:01:44,210 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/ee21a9844ef94eff933fdb727150cafc, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/51a8a0ad5aa4457e87ae4361595cae35, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/80460a63a95945e39d28e29a10d79b3c] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=26.0 M 2020-11-17 03:01:44,242 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:01:44,244 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:01:44,265 INFO [Thread-1109] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:01:49,560 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#231 average throughput is 59.11 MB/second, slept 6 time(s) and total slept time is 2460 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 03:01:49,621 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 269c6a807b824a9497459e0858306b95(size=26.9 M), total size for store is 42.6 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 03:01:49,621 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=11, startTime=1605582104210; duration=5sec 2020-11-17 03:02:01,430 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:02:07,099 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:02:07,564 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.99 MB heapSize=128.00 MB 2020-11-17 03:02:07,611 INFO [Thread-1112] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:02:08,449 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.99 MB at sequenceid=262584571 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/24a3f3c69de945378e156689fad2110b 2020-11-17 03:02:08,477 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/24a3f3c69de945378e156689fad2110b, entries=361118, sequenceid=262584571, filesize=8.1 M 2020-11-17 03:02:08,479 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.99 MB/106939771, heapSize ~128.00 MB/134222168, currentSize=35.53 KB/36382 for e5dc5c620be19b02666a19c280ff582e in 915ms, sequenceid=262584571, compaction requested=true 2020-11-17 03:02:17,243 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:02:31,393 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:02:37,260 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582049202 with entries=39645, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582157217 2020-11-17 03:03:18,061 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:03:19,636 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 196309 ns. 2020-11-17 03:03:19,636 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 11208 ns. 2020-11-17 03:03:21,775 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:03:41,639 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:03:58,906 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582157217 with entries=30079, filesize=128.55 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582238870 2020-11-17 03:04:21,242 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.94 MB heapSize=128.02 MB 2020-11-17 03:04:21,280 INFO [Thread-1121] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:04:22,004 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.94 MB at sequenceid=272598035 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/5f70c80f27eb48c294303336d0857c95 2020-11-17 03:04:22,010 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:04:22,028 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/5f70c80f27eb48c294303336d0857c95, entries=275434, sequenceid=272598035, filesize=8.5 M 2020-11-17 03:04:22,031 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~107.94 MB/113180998, heapSize ~128.02 MB/134240456, currentSize=662.69 KB/678598 for fc93f6dcc564341dcc983bf30971f73d in 790ms, sequenceid=272598035, compaction requested=true 2020-11-17 03:04:22,032 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 03:04:22,032 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/e0ee001b7f4e4061b13ade0bf33e7ad9, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/acd20cc678f54642adef84b4217812b4, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/5f70c80f27eb48c294303336d0857c95] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=25.5 M 2020-11-17 03:04:22,060 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:04:22,064 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:04:22,103 INFO [Thread-1123] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:04:22,450 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bac2756ecc315bfd457fb71d81dd3eb5 1/1 column families, dataSize=128 B heapSize=456 B 2020-11-17 03:04:22,475 INFO [Thread-1125] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:04:22,521 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=128 B at sequenceid=2032 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/__test-table_cvp23__/bac2756ecc315bfd457fb71d81dd3eb5/.tmp/__status_family___/ff379c112645408d9a0176f440590fb8 2020-11-17 03:04:22,532 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:04:22,551 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/__test-table_cvp23__/bac2756ecc315bfd457fb71d81dd3eb5/__status_family___/ff379c112645408d9a0176f440590fb8, entries=1, sequenceid=2032, filesize=5.0 K 2020-11-17 03:04:22,554 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~128 B/128, heapSize ~440 B/440, currentSize=0 B/0 for bac2756ecc315bfd457fb71d81dd3eb5 in 104ms, sequenceid=2032, compaction requested=true 2020-11-17 03:04:22,557 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of __status_family___ in __test-table_cvp23__,,1598623751841.bac2756ecc315bfd457fb71d81dd3eb5. 2020-11-17 03:04:22,557 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/__test-table_cvp23__/bac2756ecc315bfd457fb71d81dd3eb5/__status_family___/3b55506ec4da44f0bb223b43528d4150, hdfs://mycluster/hbase/data/default/__test-table_cvp23__/bac2756ecc315bfd457fb71d81dd3eb5/__status_family___/f6a126b6539d46b89d35b08cf8a6b18d, hdfs://mycluster/hbase/data/default/__test-table_cvp23__/bac2756ecc315bfd457fb71d81dd3eb5/__status_family___/ff379c112645408d9a0176f440590fb8] into tmpdir=hdfs://mycluster/hbase/data/default/__test-table_cvp23__/bac2756ecc315bfd457fb71d81dd3eb5/.tmp, totalSize=15.5 K 2020-11-17 03:04:22,588 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:04:22,590 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:04:22,597 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: bac2756ecc315bfd457fb71d81dd3eb5#__status_family___#compaction#236 average throughput is 0.37 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 03:04:22,605 INFO [Thread-1127] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:04:22,688 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 (all) file(s) in __status_family___ of bac2756ecc315bfd457fb71d81dd3eb5 into 9db0f972d318421398e24d4e5db204a4(size=5.2 K), total size for store is 5.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2020-11-17 03:04:22,689 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=__test-table_cvp23__,,1598623751841.bac2756ecc315bfd457fb71d81dd3eb5., storeName=__status_family___, priority=13, startTime=1605582262555; duration=0sec 2020-11-17 03:04:27,335 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#234 average throughput is 61.63 MB/second, slept 6 time(s) and total slept time is 2432 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 03:04:27,402 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into 383b78816aa343a4b4b6e15c2428cd24(size=26.2 M), total size for store is 33.3 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 03:04:27,402 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605582262031; duration=5sec 2020-11-17 03:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=174.93 MB, max=1.83 GB, blockCount=44148, accesses=11202336, hits=9746019, hitRatio=87.00%, , cachingAccesses=9797036, cachingHits=9743225, cachingHitsRatio=99.45%, evictions=1289, evicted=0, evictedPerRun=0.0 2020-11-17 03:04:39,538 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 67607 ns. 2020-11-17 03:04:48,769 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.38 MB heapSize=128.06 MB 2020-11-17 03:04:48,819 INFO [Thread-1130] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:04:49,502 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.38 MB at sequenceid=251620700 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/08d82902e2494805afb73a5aa97150b6 2020-11-17 03:04:49,509 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:04:49,531 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/08d82902e2494805afb73a5aa97150b6, entries=310050, sequenceid=251620700, filesize=6.4 M 2020-11-17 03:04:49,533 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.38 MB/111547641, heapSize ~128.06 MB/134279256, currentSize=69.35 KB/71013 for 82421ce8c5c8b48c573baee83caa8305 in 764ms, sequenceid=251620700, compaction requested=true 2020-11-17 03:04:49,534 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 03:04:49,534 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/68be103afc1f48d9a0bf52180b74c093, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a8ca16932f9d41bf949659a51155accc, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/08d82902e2494805afb73a5aa97150b6] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=97.2 M 2020-11-17 03:04:49,576 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:04:49,578 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:04:49,612 INFO [Thread-1132] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:05:09,157 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:05:13,397 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:05:13,401 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:05:15,516 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:05:15,520 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:05:15,712 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:05:16,059 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#238 average throughput is 59.90 MB/second, slept 29 time(s) and total slept time is 11368 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 03:05:16,130 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 2372b4e34c5d4cf5826f64cdcd0e583d(size=97.0 M), total size for store is 11.0 G. This selection was in queue for 0sec, and took 26sec to execute. 2020-11-17 03:05:16,130 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605582289533; duration=26sec 2020-11-17 03:05:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 100888 ns. 2020-11-17 03:05:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 201074 ns. 2020-11-17 03:05:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 46464 ns. 2020-11-17 03:05:47,659 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.66 MB heapSize=128.00 MB 2020-11-17 03:05:47,690 INFO [Thread-1136] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:05:48,630 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.66 MB at sequenceid=286887904 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/d6a098c062fe47e5b31e92475a081b6c 2020-11-17 03:05:48,638 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:05:48,656 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/d6a098c062fe47e5b31e92475a081b6c, entries=521387, sequenceid=286887904, filesize=9.2 M 2020-11-17 03:05:48,657 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.66 MB/95059763, heapSize ~128.00 MB/134219232, currentSize=26.18 KB/26808 for 5e5f05205d7e40b0141cd977dd009ba1 in 998ms, sequenceid=286887904, compaction requested=true 2020-11-17 03:05:48,658 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 03:05:48,659 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/ef02f2a75941473f9348d2f23a8ab098, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/270228eaa23e4bdb95c476f9aea83b4a, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/d6a098c062fe47e5b31e92475a081b6c] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=27.5 M 2020-11-17 03:05:48,692 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:05:48,696 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:05:48,730 INFO [Thread-1138] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:05:53,092 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#240 average throughput is 61.68 MB/second, slept 4 time(s) and total slept time is 647 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 03:05:53,173 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into 6afe171935cd46019c36621c834f0645(size=27.8 M), total size for store is 24.2 G. This selection was in queue for 0sec, and took 4sec to execute. 2020-11-17 03:05:53,173 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=11, startTime=1605582348658; duration=4sec 2020-11-17 03:05:57,909 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582238870 with entries=44098, filesize=128.67 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582357876 2020-11-17 03:05:57,910 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605578408212 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605578408212 2020-11-17 03:06:07,020 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=97.84 MB heapSize=128.10 MB 2020-11-17 03:06:07,048 INFO [Thread-1140] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:06:07,858 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.84 MB at sequenceid=262596730 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/345dec83023441b794f3fd84801c6589 2020-11-17 03:06:07,865 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:06:07,891 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/345dec83023441b794f3fd84801c6589, entries=421383, sequenceid=262596730, filesize=7.9 M 2020-11-17 03:06:07,894 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~97.84 MB/102591759, heapSize ~128.10 MB/134319304, currentSize=464.31 KB/475458 for e5dc5c620be19b02666a19c280ff582e in 874ms, sequenceid=262596730, compaction requested=true 2020-11-17 03:06:07,895 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 03:06:07,895 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/20cd1c26ee9a4f9bb12699cda2dbf9b6, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/24a3f3c69de945378e156689fad2110b, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/345dec83023441b794f3fd84801c6589] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=59.2 M 2020-11-17 03:06:07,926 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:06:07,928 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:06:07,952 INFO [Thread-1142] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:06:19,401 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#242 average throughput is 61.59 MB/second, slept 13 time(s) and total slept time is 4242 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 03:06:19,465 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:06:19,486 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into e0892a5aa6f042d193d3f6a07a350d10(size=60.6 M), total size for store is 48.7 G. This selection was in queue for 0sec, and took 11sec to execute. 2020-11-17 03:06:19,486 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605582367894; duration=11sec 2020-11-17 03:06:47,976 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:07:06,117 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582357876 with entries=25358, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582426082 2020-11-17 03:07:39,542 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 58721 ns. 2020-11-17 03:08:10,405 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:08:28,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18046 ns. 2020-11-17 03:08:28,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 32065 ns. 2020-11-17 03:09:05,187 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582426082 with entries=44125, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582545131 2020-11-17 03:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=176.11 MB, max=1.83 GB, blockCount=44115, accesses=11246757, hits=9748843, hitRatio=86.68%, , cachingAccesses=9799842, cachingHits=9746024, cachingHitsRatio=99.45%, evictions=1319, evicted=0, evictedPerRun=0.0 2020-11-17 03:09:38,838 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.20 MB heapSize=128.00 MB 2020-11-17 03:09:38,878 INFO [Thread-1157] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:09:39,597 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.20 MB at sequenceid=262607494 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/08590a4b835a4472a291a3ad7a4088ef 2020-11-17 03:09:39,602 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:09:39,625 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/08590a4b835a4472a291a3ad7a4088ef, entries=371965, sequenceid=262607494, filesize=8.2 M 2020-11-17 03:09:39,626 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.20 MB/106112979, heapSize ~128.00 MB/134217776, currentSize=472.49 KB/483827 for e5dc5c620be19b02666a19c280ff582e in 788ms, sequenceid=262607494, compaction requested=true 2020-11-17 03:09:39,627 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 03:09:39,628 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/0367aebceb5f4d13a159c58ab1ce66a2, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/59a81f420aef43d7a34781eff8f52ac3, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e0892a5aa6f042d193d3f6a07a350d10, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/08590a4b835a4472a291a3ad7a4088ef] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=394.2 M 2020-11-17 03:09:39,672 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:09:39,676 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:09:39,689 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:09:39,703 INFO [Thread-1159] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:09:48,725 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:09:55,536 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:09:59,464 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:10:03,684 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/d2d14e2dd9bc4ed2bb245097cd27da94] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:10:15,172 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] compress.CodecPool: Got brand-new compressor [.snappy] 2020-11-17 03:10:20,582 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582545131 with entries=27817, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582620539 2020-11-17 03:10:30,081 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/d2d14e2dd9bc4ed2bb245097cd27da94] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:10:38,658 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 108069 ns. 2020-11-17 03:10:57,355 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:10:57,431 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:10:57,930 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/d2d14e2dd9bc4ed2bb245097cd27da94] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:10:58,569 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:10:59,624 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:10:59,709 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:11:00,689 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#244 average throughput is 55.88 MB/second, slept 81 time(s) and total slept time is 27522 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 03:11:00,716 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.49 MB heapSize=128.00 MB 2020-11-17 03:11:00,763 INFO [Thread-1167] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:11:00,786 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of e5dc5c620be19b02666a19c280ff582e into d2d14e2dd9bc4ed2bb245097cd27da94(size=396.7 M), total size for store is 48.7 G. This selection was in queue for 0sec, and took 1mins, 21sec to execute. 2020-11-17 03:11:00,786 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605582579627; duration=1mins, 21sec 2020-11-17 03:11:01,956 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.49 MB at sequenceid=261106683 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/3664bc7ef11343a7b10b71951a045774 2020-11-17 03:11:01,995 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/3664bc7ef11343a7b10b71951a045774, entries=576118, sequenceid=261106683, filesize=8.9 M 2020-11-17 03:11:01,996 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.49 MB/89639183, heapSize ~128.00 MB/134218984, currentSize=76.13 KB/77957 for 73aae5715542e1a5370f7540d2117abc in 1281ms, sequenceid=261106683, compaction requested=true 2020-11-17 03:11:19,753 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.15 MB heapSize=128.06 MB 2020-11-17 03:11:19,809 INFO [Thread-1170] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:11:20,460 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.15 MB at sequenceid=251632183 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/ae69c44583d9431ea623816f4741b59c 2020-11-17 03:11:20,467 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:11:20,484 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/ae69c44583d9431ea623816f4741b59c, entries=313162, sequenceid=251632183, filesize=6.4 M 2020-11-17 03:11:20,486 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.15 MB/111302622, heapSize ~128.06 MB/134279608, currentSize=153.22 KB/156896 for 82421ce8c5c8b48c573baee83caa8305 in 733ms, sequenceid=251632183, compaction requested=true 2020-11-17 03:11:23,289 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.51 MB heapSize=128.08 MB 2020-11-17 03:11:23,422 INFO [Thread-1172] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:11:24,252 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.51 MB at sequenceid=259264722 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/ae15afb46cac46ba846a1142c23b67a7 2020-11-17 03:11:24,259 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:11:24,278 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/ae15afb46cac46ba846a1142c23b67a7, entries=381754, sequenceid=259264722, filesize=7.3 M 2020-11-17 03:11:24,284 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.51 MB/105387133, heapSize ~128.08 MB/134296568, currentSize=97.98 KB/100333 for 06f7ca9534710e37d90d9f8345864259 in 995ms, sequenceid=259264722, compaction requested=true 2020-11-17 03:11:38,743 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 192327 ns. 2020-11-17 03:11:38,743 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 46418 ns. 2020-11-17 03:11:38,743 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 60225 ns. 2020-11-17 03:11:46,576 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:12:13,636 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:12:16,545 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582620539 with entries=42729, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582736499 2020-11-17 03:12:21,357 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:13:00,370 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.12 MB heapSize=128.05 MB 2020-11-17 03:13:00,400 INFO [Thread-1179] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:13:01,309 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.12 MB at sequenceid=262617537 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/01b39787c7b94775ac25ed5cfb21c9be 2020-11-17 03:13:01,316 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:13:01,355 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/01b39787c7b94775ac25ed5cfb21c9be, entries=359738, sequenceid=262617537, filesize=8.2 M 2020-11-17 03:13:01,357 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.12 MB/107077860, heapSize ~128.05 MB/134274704, currentSize=1.04 MB/1090977 for e5dc5c620be19b02666a19c280ff582e in 987ms, sequenceid=262617537, compaction requested=true 2020-11-17 03:13:17,758 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:13:28,984 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 45208 ns. 2020-11-17 03:13:28,984 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 11087 ns. 2020-11-17 03:13:31,988 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:13:33,260 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582736499 with entries=27880, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582813231 2020-11-17 03:13:39,494 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 56005 ns. 2020-11-17 03:13:41,033 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:13:58,418 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:14:11,078 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:14:34,147 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=176.72 MB, max=1.83 GB, blockCount=44089, accesses=11567442, hits=9996001, hitRatio=86.41%, , cachingAccesses=10046987, cachingHits=9993129, cachingHitsRatio=99.46%, evictions=1349, evicted=0, evictedPerRun=0.0 2020-11-17 03:15:10,271 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:15:21,783 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.76 MB heapSize=128.00 MB 2020-11-17 03:15:21,827 INFO [Thread-1186] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:15:23,127 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.76 MB at sequenceid=290808587 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/849815c9112744eeaaa6d439ba6acb50 2020-11-17 03:15:23,134 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:15:23,152 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/849815c9112744eeaaa6d439ba6acb50, entries=545713, sequenceid=290808587, filesize=9.1 M 2020-11-17 03:15:23,154 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~88.76 MB/93072527, heapSize ~128.00 MB/134218672, currentSize=74.59 KB/76381 for bdaa31a63b7f3d06d9909b7560333e79 in 1372ms, sequenceid=290808587, compaction requested=true 2020-11-17 03:15:24,256 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582813231 with entries=42157, filesize=128.18 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582924224 2020-11-17 03:15:24,257 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580770052 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605580770052 2020-11-17 03:15:24,277 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605580916608 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605580916608 2020-11-17 03:15:24,283 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581040333 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605581040333 2020-11-17 03:15:24,290 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581153689 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605581153689 2020-11-17 03:15:24,297 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581304916 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605581304916 2020-11-17 03:15:24,302 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581441959 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605581441959 2020-11-17 03:15:24,308 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581548915 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605581548915 2020-11-17 03:15:24,316 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581669859 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605581669859 2020-11-17 03:15:24,322 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581772752 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605581772752 2020-11-17 03:15:24,327 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581856079 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605581856079 2020-11-17 03:15:24,332 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605581965372 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605581965372 2020-11-17 03:15:35,682 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.91 MB heapSize=128.09 MB 2020-11-17 03:15:35,725 INFO [Thread-1189] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:15:36,463 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.91 MB at sequenceid=271437955 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/b35b52264143467492f87704b6f812b1 2020-11-17 03:15:36,470 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:15:36,494 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/b35b52264143467492f87704b6f812b1, entries=318720, sequenceid=271437955, filesize=8.7 M 2020-11-17 03:15:36,496 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.91 MB/110007441, heapSize ~128.09 MB/134313016, currentSize=831.97 KB/851936 for 0469ce862509756add4a8cdb00bc465c in 815ms, sequenceid=271437955, compaction requested=true 2020-11-17 03:15:38,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 94082 ns. 2020-11-17 03:15:56,284 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:16:08,723 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:16:39,704 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582924224 with entries=27821, filesize=128.24 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582999658 2020-11-17 03:16:39,705 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582049202 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605582049202 2020-11-17 03:16:39,714 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582157217 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605582157217 2020-11-17 03:16:47,984 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:16:59,581 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.25 MB heapSize=128.16 MB 2020-11-17 03:16:59,621 INFO [Thread-1196] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:17:00,319 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.25 MB at sequenceid=262629705 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/1084eb9a8f204aacaa64fd83ed65e36c 2020-11-17 03:17:00,325 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:17:00,343 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1084eb9a8f204aacaa64fd83ed65e36c, entries=359379, sequenceid=262629705, filesize=8.1 M 2020-11-17 03:17:00,344 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.25 MB/107214824, heapSize ~128.16 MB/134388968, currentSize=1.24 MB/1299391 for e5dc5c620be19b02666a19c280ff582e in 764ms, sequenceid=262629705, compaction requested=true 2020-11-17 03:17:30,864 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.19 MB heapSize=128.08 MB 2020-11-17 03:17:30,908 INFO [Thread-1199] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:17:31,628 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.19 MB at sequenceid=251643132 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/75094e78f39147789a8ff25cedfe7f17 2020-11-17 03:17:31,634 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:17:31,650 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/75094e78f39147789a8ff25cedfe7f17, entries=312739, sequenceid=251643132, filesize=6.4 M 2020-11-17 03:17:31,652 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.19 MB/111349596, heapSize ~128.08 MB/134298256, currentSize=20.15 KB/20638 for 82421ce8c5c8b48c573baee83caa8305 in 788ms, sequenceid=251643132, compaction requested=true 2020-11-17 03:17:31,653 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 03:17:31,653 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/2372b4e34c5d4cf5826f64cdcd0e583d, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/ae69c44583d9431ea623816f4741b59c, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/75094e78f39147789a8ff25cedfe7f17] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=109.8 M 2020-11-17 03:17:31,683 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:17:31,686 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:17:31,718 INFO [Thread-1201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:17:38,778 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 51066 ns. 2020-11-17 03:17:38,778 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 54950 ns. 2020-11-17 03:17:38,779 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 175118 ns. 2020-11-17 03:17:45,443 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:17:58,763 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:18:01,967 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:18:02,431 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:18:04,937 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:18:05,546 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#253 average throughput is 53.01 MB/second, slept 29 time(s) and total slept time is 12277 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 03:18:05,614 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into f16a41b6b30949b288171f38df401087(size=109.5 M), total size for store is 11.0 G. This selection was in queue for 0sec, and took 33sec to execute. 2020-11-17 03:18:05,614 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605583051652; duration=33sec 2020-11-17 03:18:15,610 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:18:25,472 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582999658 with entries=39382, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583105430 2020-11-17 03:18:38,260 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:18:38,974 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15347 ns. 2020-11-17 03:18:38,974 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 38009 ns. 2020-11-17 03:18:53,720 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=175.84 MB, max=1.83 GB, blockCount=44110, accesses=11602472, hits=10001934, hitRatio=86.21%, , cachingAccesses=10052938, cachingHits=9999059, cachingHitsRatio=99.46%, evictions=1379, evicted=0, evictedPerRun=0.0 2020-11-17 03:19:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 48182 ns. 2020-11-17 03:19:43,182 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:19:48,628 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583105430 with entries=30566, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583188600 2020-11-17 03:19:52,489 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.84 MB heapSize=128.00 MB 2020-11-17 03:19:52,513 INFO [Thread-1209] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:19:53,274 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.84 MB at sequenceid=272626489 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/4421e61511244e308d688eb128f52c7e 2020-11-17 03:19:53,281 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:19:53,307 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/4421e61511244e308d688eb128f52c7e, entries=276606, sequenceid=272626489, filesize=8.5 M 2020-11-17 03:19:53,309 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~107.84 MB/113080357, heapSize ~128.00 MB/134220272, currentSize=14.91 KB/15271 for fc93f6dcc564341dcc983bf30971f73d in 820ms, sequenceid=272626489, compaction requested=true 2020-11-17 03:21:18,652 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.13 MB heapSize=128.11 MB 2020-11-17 03:21:18,691 INFO [Thread-1214] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:21:19,461 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.13 MB at sequenceid=262642894 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/44119d76805140ca82d468792d9bc06c 2020-11-17 03:21:19,473 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:21:19,497 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/44119d76805140ca82d468792d9bc06c, entries=417484, sequenceid=262642894, filesize=8.1 M 2020-11-17 03:21:19,499 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~98.13 MB/102896969, heapSize ~128.11 MB/134334304, currentSize=528.94 KB/541638 for e5dc5c620be19b02666a19c280ff582e in 848ms, sequenceid=262642894, compaction requested=true 2020-11-17 03:21:19,500 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 03:21:19,500 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/01b39787c7b94775ac25ed5cfb21c9be, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1084eb9a8f204aacaa64fd83ed65e36c, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/44119d76805140ca82d468792d9bc06c] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.4 M 2020-11-17 03:21:19,539 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:21:19,541 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:21:19,566 INFO [Thread-1216] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:21:20,185 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:21:24,801 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#256 average throughput is 57.58 MB/second, slept 6 time(s) and total slept time is 1550 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 03:21:24,911 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into b14e9ea4bd26413cac4bb74321da2217(size=25.1 M), total size for store is 48.7 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 03:21:24,911 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605583279499; duration=5sec 2020-11-17 03:21:38,719 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 103324 ns. 2020-11-17 03:21:41,142 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583188600 with entries=41495, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583301115 2020-11-17 03:22:38,612 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:23:07,762 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583301115 with entries=32310, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583387725 2020-11-17 03:23:14,770 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:23:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 63043 ns. 2020-11-17 03:23:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 56471 ns. 2020-11-17 03:23:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 240471 ns. 2020-11-17 03:23:48,731 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 11639 ns. 2020-11-17 03:23:48,731 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 44480 ns. 2020-11-17 03:23:55,034 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.71 MB heapSize=128.03 MB 2020-11-17 03:23:55,072 INFO [Thread-1226] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:23:55,929 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.71 MB at sequenceid=259303922 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/391d8fa80b2947a7a616ea0783a897aa 2020-11-17 03:23:55,936 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:23:55,963 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/391d8fa80b2947a7a616ea0783a897aa, entries=383055, sequenceid=259303922, filesize=7.3 M 2020-11-17 03:23:55,964 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.71 MB/105604391, heapSize ~128.03 MB/134248984, currentSize=127.37 KB/130431 for 06f7ca9534710e37d90d9f8345864259 in 931ms, sequenceid=259303922, compaction requested=true 2020-11-17 03:23:55,966 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 03:23:55,966 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/eae24701c9bb449fb4f6477235fc6c6a, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/ae15afb46cac46ba846a1142c23b67a7, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/391d8fa80b2947a7a616ea0783a897aa] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=52.3 M 2020-11-17 03:23:55,994 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:23:56,001 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:23:56,024 INFO [Thread-1228] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:23:56,120 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:24:07,708 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#258 average throughput is 59.53 MB/second, slept 13 time(s) and total slept time is 4850 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 03:24:07,794 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 7861dd70448f42e39ce51d0b206f75f1(size=52.7 M), total size for store is 55.3 G. This selection was in queue for 0sec, and took 11sec to execute. 2020-11-17 03:24:07,794 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605583435965; duration=11sec 2020-11-17 03:24:19,256 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.11 MB heapSize=128.14 MB 2020-11-17 03:24:19,301 INFO [Thread-1232] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:24:20,010 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.11 MB at sequenceid=251655134 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/2f1263e536ab44b2bc1ada5099f13aa2 2020-11-17 03:24:20,017 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:24:20,031 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/2f1263e536ab44b2bc1ada5099f13aa2, entries=314771, sequenceid=251655134, filesize=6.4 M 2020-11-17 03:24:20,032 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.11 MB/111260627, heapSize ~128.14 MB/134362576, currentSize=850.04 KB/870442 for 82421ce8c5c8b48c573baee83caa8305 in 776ms, sequenceid=251655134, compaction requested=true 2020-11-17 03:24:27,913 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:24:32,333 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=176.09 MB, max=1.83 GB, blockCount=44109, accesses=11626218, hits=10009436, hitRatio=86.09%, , cachingAccesses=10060436, cachingHits=10006547, cachingHitsRatio=99.46%, evictions=1409, evicted=0, evictedPerRun=0.0 2020-11-17 03:24:48,914 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.75 MB heapSize=128.31 MB 2020-11-17 03:24:48,984 INFO [Thread-1235] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:24:49,695 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.75 MB at sequenceid=262653467 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/98c79e33b5fd4b85a1c92183516f10c1 2020-11-17 03:24:49,701 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:24:49,719 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/98c79e33b5fd4b85a1c92183516f10c1, entries=382767, sequenceid=262653467, filesize=8.2 M 2020-11-17 03:24:49,720 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.75 MB/105643251, heapSize ~128.31 MB/134547120, currentSize=726.59 KB/744031 for e5dc5c620be19b02666a19c280ff582e in 806ms, sequenceid=262653467, compaction requested=true 2020-11-17 03:24:50,007 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583387725 with entries=37988, filesize=128.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583489933 2020-11-17 03:24:57,206 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:25:01,016 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=84.66 MB heapSize=128.00 MB 2020-11-17 03:25:01,043 INFO [Thread-1237] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:25:02,105 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=84.66 MB at sequenceid=261150488 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/a92e51d1689a4159a74f520db6c0e616 2020-11-17 03:25:02,112 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:25:02,136 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/a92e51d1689a4159a74f520db6c0e616, entries=579616, sequenceid=261150488, filesize=8.8 M 2020-11-17 03:25:02,137 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~84.66 MB/88775071, heapSize ~128.00 MB/134218768, currentSize=104.11 KB/106613 for 73aae5715542e1a5370f7540d2117abc in 1121ms, sequenceid=261150488, compaction requested=true 2020-11-17 03:25:02,138 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 03:25:02,138 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/f814e36eeea54109bf5e13b3e19c53ae, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/3664bc7ef11343a7b10b71951a045774, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/a92e51d1689a4159a74f520db6c0e616] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=26.5 M 2020-11-17 03:25:02,176 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:25:02,181 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:25:02,208 INFO [Thread-1239] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:25:02,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:25:06,123 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#262 average throughput is 63.18 MB/second, slept 3 time(s) and total slept time is 212 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 03:25:06,191 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 83924e9b4f9e4aa1848516bb502a50ea(size=27.0 M), total size for store is 12.5 G. This selection was in queue for 0sec, and took 4sec to execute. 2020-11-17 03:25:06,192 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605583502137; duration=4sec 2020-11-17 03:25:18,804 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:25:38,770 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 83006 ns. 2020-11-17 03:26:20,695 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:26:27,175 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583489933 with entries=35659, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583587147 2020-11-17 03:26:47,990 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:27:38,795 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 107092 ns. 2020-11-17 03:27:57,985 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583587147 with entries=33372, filesize=128.26 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583677946 2020-11-17 03:28:03,637 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.51 MB heapSize=128.36 MB 2020-11-17 03:28:03,679 INFO [Thread-1249] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:28:04,552 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.51 MB at sequenceid=262663118 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a9277eaa26f04c1494c7bbc927e5f04b 2020-11-17 03:28:04,569 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:28:04,590 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a9277eaa26f04c1494c7bbc927e5f04b, entries=358394, sequenceid=262663118, filesize=8.2 M 2020-11-17 03:28:04,592 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.51 MB/107494075, heapSize ~128.36 MB/134595360, currentSize=1.22 MB/1279330 for e5dc5c620be19b02666a19c280ff582e in 956ms, sequenceid=262663118, compaction requested=true 2020-11-17 03:28:04,593 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 03:28:04,594 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b14e9ea4bd26413cac4bb74321da2217, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/98c79e33b5fd4b85a1c92183516f10c1, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a9277eaa26f04c1494c7bbc927e5f04b] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.5 M 2020-11-17 03:28:04,640 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:28:04,647 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:28:04,678 INFO [Thread-1251] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:28:06,248 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:28:13,427 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#264 average throughput is 57.69 MB/second, slept 9 time(s) and total slept time is 2426 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 03:28:13,552 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 51208e877bce49aea0caea9b3a427f79(size=42.6 M), total size for store is 48.8 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 03:28:13,552 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605583684593; duration=8sec 2020-11-17 03:28:49,617 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 31551 ns. 2020-11-17 03:28:49,617 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 37253 ns. 2020-11-17 03:29:10,686 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.09 MB heapSize=128.00 MB 2020-11-17 03:29:10,739 INFO [Thread-1256] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:29:11,669 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.09 MB at sequenceid=271477426 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/7e510599a4da4f7480f0ac08c59b80b2 2020-11-17 03:29:11,674 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:29:11,708 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/7e510599a4da4f7480f0ac08c59b80b2, entries=315674, sequenceid=271477426, filesize=8.7 M 2020-11-17 03:29:11,710 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.09 MB/110191551, heapSize ~128.00 MB/134218448, currentSize=76.63 KB/78472 for 0469ce862509756add4a8cdb00bc465c in 1024ms, sequenceid=271477426, compaction requested=true 2020-11-17 03:29:11,711 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 03:29:11,711 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/269c6a807b824a9497459e0858306b95, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/b35b52264143467492f87704b6f812b1, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/7e510599a4da4f7480f0ac08c59b80b2] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=44.3 M 2020-11-17 03:29:11,741 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:29:11,746 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:29:11,770 INFO [Thread-1258] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:29:20,591 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#266 average throughput is 59.21 MB/second, slept 10 time(s) and total slept time is 3942 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 03:29:20,656 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 7d95767f650e44e6981473b78766033d(size=45.2 M), total size for store is 42.6 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 03:29:20,656 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=11, startTime=1605583751711; duration=8sec 2020-11-17 03:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=175.56 MB, max=1.83 GB, blockCount=44115, accesses=11897309, hits=10259720, hitRatio=86.24%, , cachingAccesses=10310740, cachingHits=10256808, cachingHitsRatio=99.48%, evictions=1439, evicted=0, evictedPerRun=0.0 2020-11-17 03:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 03:29:39,157 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 301682 ns. 2020-11-17 03:29:39,158 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 62777 ns. 2020-11-17 03:29:39,158 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 52476 ns. 2020-11-17 03:29:45,136 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583677946 with entries=39185, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583785111 2020-11-17 03:30:08,720 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.25 MB heapSize=128.01 MB 2020-11-17 03:30:08,764 INFO [Thread-1263] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:30:09,927 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.25 MB at sequenceid=286967674 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/3d26705a67a94019ae10daf8b6ca9ed0 2020-11-17 03:30:09,932 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:30:09,949 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/3d26705a67a94019ae10daf8b6ca9ed0, entries=527611, sequenceid=286967674, filesize=9.1 M 2020-11-17 03:30:09,950 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.25 MB/94628962, heapSize ~128.01 MB/134228424, currentSize=314.92 KB/322479 for 5e5f05205d7e40b0141cd977dd009ba1 in 1230ms, sequenceid=286967674, compaction requested=true 2020-11-17 03:30:22,200 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:30:35,533 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:30:38,943 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 50534 ns. 2020-11-17 03:30:48,211 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:30:56,950 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:02,415 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.01 MB heapSize=128.01 MB 2020-11-17 03:31:02,447 INFO [Thread-1267] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:03,161 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.01 MB at sequenceid=251666871 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/354d052bd50e4c878e587e19f8044f7a 2020-11-17 03:31:03,169 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:03,194 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/354d052bd50e4c878e587e19f8044f7a, entries=314295, sequenceid=251666871, filesize=6.4 M 2020-11-17 03:31:03,196 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.01 MB/111156606, heapSize ~128.01 MB/134222992, currentSize=22.62 KB/23166 for 82421ce8c5c8b48c573baee83caa8305 in 781ms, sequenceid=251666871, compaction requested=true 2020-11-17 03:31:03,197 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 03:31:03,197 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/f16a41b6b30949b288171f38df401087, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/2f1263e536ab44b2bc1ada5099f13aa2, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/354d052bd50e4c878e587e19f8044f7a] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=122.4 M 2020-11-17 03:31:03,233 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:03,237 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:03,266 INFO [Thread-1269] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:03,385 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:05,018 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583785111 with entries=29886, filesize=128.34 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583864989 2020-11-17 03:31:05,019 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582238870 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605582238870 2020-11-17 03:31:05,031 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582357876 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605582357876 2020-11-17 03:31:05,037 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582426082 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605582426082 2020-11-17 03:31:05,047 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582545131 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605582545131 2020-11-17 03:31:05,052 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582620539 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605582620539 2020-11-17 03:31:05,056 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582736499 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605582736499 2020-11-17 03:31:10,863 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:14,478 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:18,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:23,895 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:29,433 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:32,593 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:35,419 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:36,111 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:36,168 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:36,896 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:38,747 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:39,147 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:39,543 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#269 average throughput is 55.17 MB/second, slept 37 time(s) and total slept time is 15242 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 03:31:39,632 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 6b4cf44a87a6405f8670d808835ced56(size=122.0 M), total size for store is 11.0 G. This selection was in queue for 0sec, and took 36sec to execute. 2020-11-17 03:31:39,632 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605583863196; duration=36sec 2020-11-17 03:31:44,766 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:31:54,175 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:32:01,493 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:32:02,973 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.77 MB heapSize=128.61 MB 2020-11-17 03:32:03,024 INFO [Thread-1275] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:32:03,886 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.77 MB at sequenceid=262675080 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/dd29c46a9bec4114b132effc5a939e26 2020-11-17 03:32:03,912 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/dd29c46a9bec4114b132effc5a939e26, entries=358356, sequenceid=262675080, filesize=8.3 M 2020-11-17 03:32:03,913 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.77 MB/107766106, heapSize ~128.61 MB/134861168, currentSize=978.63 KB/1002122 for e5dc5c620be19b02666a19c280ff582e in 940ms, sequenceid=262675080, compaction requested=true 2020-11-17 03:32:28,227 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:32:32,436 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:32:39,230 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 99109 ns. 2020-11-17 03:32:44,694 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:33:00,701 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583864989 with entries=42146, filesize=128.09 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583980608 2020-11-17 03:33:12,913 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:33:18,372 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:33:22,555 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:33:36,051 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:33:40,495 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:33:58,763 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 40196 ns. 2020-11-17 03:33:58,763 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 9325 ns. 2020-11-17 03:34:13,482 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583980608 with entries=27609, filesize=128.19 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584053405 2020-11-17 03:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.74 MB, max=1.83 GB, blockCount=44160, accesses=12599352, hits=10929240, hitRatio=86.74%, , cachingAccesses=10980330, cachingHits=10926323, cachingHitsRatio=99.51%, evictions=1469, evicted=0, evictedPerRun=0.0 2020-11-17 03:35:15,555 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.32 MB heapSize=128.55 MB 2020-11-17 03:35:15,595 INFO [Thread-1286] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:35:16,327 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.32 MB at sequenceid=272654567 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/3f74d15d897947718f7e94196a653ba5 2020-11-17 03:35:16,333 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:35:16,351 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/3f74d15d897947718f7e94196a653ba5, entries=277492, sequenceid=272654567, filesize=8.5 M 2020-11-17 03:35:16,352 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.32 MB/113585317, heapSize ~128.55 MB/134794032, currentSize=13.12 KB/13432 for fc93f6dcc564341dcc983bf30971f73d in 797ms, sequenceid=272654567, compaction requested=true 2020-11-17 03:35:16,353 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 03:35:16,353 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/383b78816aa343a4b4b6e15c2428cd24, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/4421e61511244e308d688eb128f52c7e, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/3f74d15d897947718f7e94196a653ba5] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=43.1 M 2020-11-17 03:35:16,380 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:35:16,385 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:35:16,405 INFO [Thread-1288] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:35:25,440 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#272 average throughput is 59.64 MB/second, slept 10 time(s) and total slept time is 3974 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 03:35:25,515 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into d49077802f5d4db7ad896a2b67f75b42(size=43.7 M), total size for store is 33.3 G. This selection was in queue for 0sec, and took 9sec to execute. 2020-11-17 03:35:25,515 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605584116352; duration=9sec 2020-11-17 03:35:38,739 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 220701 ns. 2020-11-17 03:35:38,740 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 49438 ns. 2020-11-17 03:35:38,740 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 68175 ns. 2020-11-17 03:36:03,405 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584053405 with entries=40344, filesize=128.49 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584163377 2020-11-17 03:36:10,163 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.69 MB heapSize=128.05 MB 2020-11-17 03:36:10,199 INFO [Thread-1292] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:36:11,031 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.69 MB at sequenceid=259342144 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/72fdc106413a4668bf9746873e2aab38 2020-11-17 03:36:11,040 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:36:11,058 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/72fdc106413a4668bf9746873e2aab38, entries=381141, sequenceid=259342144, filesize=7.3 M 2020-11-17 03:36:11,059 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.69 MB/105584781, heapSize ~128.05 MB/134269224, currentSize=80.97 KB/82914 for 06f7ca9534710e37d90d9f8345864259 in 896ms, sequenceid=259342144, compaction requested=true 2020-11-17 03:36:25,453 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=97.94 MB heapSize=128.36 MB 2020-11-17 03:36:25,518 INFO [Thread-1295] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:36:26,349 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.94 MB at sequenceid=262688398 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/eb8f2a13373543a38c17ce8c6131ad69 2020-11-17 03:36:26,379 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:36:26,397 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/eb8f2a13373543a38c17ce8c6131ad69, entries=423511, sequenceid=262688398, filesize=7.9 M 2020-11-17 03:36:26,398 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~97.94 MB/102695693, heapSize ~128.36 MB/134597480, currentSize=431.59 KB/441948 for e5dc5c620be19b02666a19c280ff582e in 945ms, sequenceid=262688398, compaction requested=true 2020-11-17 03:36:26,399 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 03:36:26,399 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/51208e877bce49aea0caea9b3a427f79, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/dd29c46a9bec4114b132effc5a939e26, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/eb8f2a13373543a38c17ce8c6131ad69] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=58.9 M 2020-11-17 03:36:26,425 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:36:26,427 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:36:26,460 INFO [Thread-1297] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:36:31,208 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:36:38,752 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#275 average throughput is 57.39 MB/second, slept 13 time(s) and total slept time is 4970 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 03:36:38,816 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 70178 ns. 2020-11-17 03:36:38,831 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:36:38,853 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into f053dbf698b74c31bd6d36e5ce60c05b(size=60.5 M), total size for store is 48.8 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 03:36:38,853 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605584186398; duration=12sec 2020-11-17 03:36:47,982 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:36:51,463 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.46 MB heapSize=128.38 MB 2020-11-17 03:36:51,508 INFO [Thread-1302] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:36:52,431 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.46 MB at sequenceid=251677188 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/96c4794418d34a24893e482158d048ed 2020-11-17 03:36:52,436 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:36:52,453 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/96c4794418d34a24893e482158d048ed, entries=313269, sequenceid=251677188, filesize=6.4 M 2020-11-17 03:36:52,454 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.46 MB/111634849, heapSize ~128.38 MB/134616440, currentSize=53.23 KB/54512 for 82421ce8c5c8b48c573baee83caa8305 in 991ms, sequenceid=251677188, compaction requested=true 2020-11-17 03:37:12,497 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584163377 with entries=25992, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584232468 2020-11-17 03:38:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 109672 ns. 2020-11-17 03:39:02,810 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584232468 with entries=40350, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584342770 2020-11-17 03:39:08,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 48156 ns. 2020-11-17 03:39:08,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 19032 ns. 2020-11-17 03:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=174.38 MB, max=1.83 GB, blockCount=44142, accesses=12622364, hits=10932068, hitRatio=86.61%, , cachingAccesses=10983151, cachingHits=10929136, cachingHitsRatio=99.51%, evictions=1499, evicted=0, evictedPerRun=0.0 2020-11-17 03:39:54,516 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.91 MB heapSize=128.00 MB 2020-11-17 03:39:54,557 INFO [Thread-1312] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:39:55,292 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.91 MB at sequenceid=262698915 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/3ba7f647b08e4966b4aa3285f8cbb7f9 2020-11-17 03:39:55,306 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:39:55,325 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3ba7f647b08e4966b4aa3285f8cbb7f9, entries=376048, sequenceid=262698915, filesize=8.2 M 2020-11-17 03:39:55,327 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.91 MB/105808964, heapSize ~128.00 MB/134220216, currentSize=695.79 KB/712490 for e5dc5c620be19b02666a19c280ff582e in 810ms, sequenceid=262698915, compaction requested=true 2020-11-17 03:40:16,472 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584342770 with entries=27663, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584416445 2020-11-17 03:40:32,107 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.71 MB heapSize=128.00 MB 2020-11-17 03:40:32,142 INFO [Thread-1315] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:40:33,220 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.71 MB at sequenceid=261198893 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/71f01d6bb3594096a827294f16a75dfc 2020-11-17 03:40:33,226 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:40:33,243 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/71f01d6bb3594096a827294f16a75dfc, entries=576884, sequenceid=261198893, filesize=9.0 M 2020-11-17 03:40:33,244 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.71 MB/89875682, heapSize ~128.00 MB/134219256, currentSize=49.79 KB/50983 for 73aae5715542e1a5370f7540d2117abc in 1137ms, sequenceid=261198893, compaction requested=true 2020-11-17 03:40:41,878 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:40:45,907 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:41:39,386 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 55249 ns. 2020-11-17 03:41:39,386 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 70593 ns. 2020-11-17 03:41:39,387 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 175192 ns. 2020-11-17 03:41:39,387 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 73511 ns. 2020-11-17 03:42:04,870 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584416445 with entries=39014, filesize=128.09 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584524834 2020-11-17 03:42:16,128 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.24 MB heapSize=128.04 MB 2020-11-17 03:42:16,182 INFO [Thread-1322] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:42:16,919 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.24 MB at sequenceid=251686795 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/d7f2734f0a4a46c886f57e80337c5741 2020-11-17 03:42:16,925 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:42:16,943 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d7f2734f0a4a46c886f57e80337c5741, entries=311620, sequenceid=251686795, filesize=6.4 M 2020-11-17 03:42:16,954 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.24 MB/111399842, heapSize ~128.04 MB/134261160, currentSize=824.47 KB/844255 for 82421ce8c5c8b48c573baee83caa8305 in 827ms, sequenceid=251686795, compaction requested=true 2020-11-17 03:42:47,379 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584524834 with entries=15802, filesize=129.62 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584567349 2020-11-17 03:42:47,401 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.25 MB heapSize=128.28 MB 2020-11-17 03:42:47,426 INFO [Thread-1325] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:42:48,248 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.25 MB at sequenceid=271517008 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/4bc268c95caa493b92cf0db92e20b3e1 2020-11-17 03:42:48,267 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:42:48,345 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/4bc268c95caa493b92cf0db92e20b3e1, entries=316435, sequenceid=271517008, filesize=8.7 M 2020-11-17 03:42:48,347 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.25 MB/110363155, heapSize ~128.28 MB/134506864, currentSize=20.42 KB/20915 for 0469ce862509756add4a8cdb00bc465c in 946ms, sequenceid=271517008, compaction requested=true 2020-11-17 03:42:54,010 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:43:01,201 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:43:16,974 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.75 MB heapSize=128.04 MB 2020-11-17 03:43:17,029 INFO [Thread-1328] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:43:17,894 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.75 MB at sequenceid=262709028 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/aa492df373144762936d23d32f39a607 2020-11-17 03:43:17,903 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:43:17,927 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/aa492df373144762936d23d32f39a607, entries=364282, sequenceid=262709028, filesize=8.2 M 2020-11-17 03:43:17,929 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.75 MB/106691909, heapSize ~128.04 MB/134255000, currentSize=1022.01 KB/1046534 for e5dc5c620be19b02666a19c280ff582e in 955ms, sequenceid=262709028, compaction requested=true 2020-11-17 03:43:17,930 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 03:43:17,930 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f053dbf698b74c31bd6d36e5ce60c05b, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3ba7f647b08e4966b4aa3285f8cbb7f9, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/aa492df373144762936d23d32f39a607] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=76.9 M 2020-11-17 03:43:17,987 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:43:17,989 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:43:18,019 INFO [Thread-1330] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:43:18,054 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:43:33,939 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#282 average throughput is 57.04 MB/second, slept 17 time(s) and total slept time is 5307 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 03:43:34,003 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 4638fdd37efb4dc6a7d80dc00f7b6f35(size=78.1 M), total size for store is 48.8 G. This selection was in queue for 0sec, and took 16sec to execute. 2020-11-17 03:43:34,003 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605584597929; duration=16sec 2020-11-17 03:43:38,798 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 107295 ns. 2020-11-17 03:44:09,244 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 41811 ns. 2020-11-17 03:44:09,244 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18566 ns. 2020-11-17 03:44:16,546 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584567349 with entries=33075, filesize=128.13 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584656513 2020-11-17 03:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.96 MB, max=1.83 GB, blockCount=44153, accesses=12951816, hits=11246733, hitRatio=86.84%, , cachingAccesses=11297831, cachingHits=11243797, cachingHitsRatio=99.52%, evictions=1529, evicted=0, evictedPerRun=0.0 2020-11-17 03:44:56,175 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:45:11,855 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:45:36,677 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.15 MB heapSize=128.00 MB 2020-11-17 03:45:36,715 INFO [Thread-1337] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:45:37,455 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.15 MB at sequenceid=259372025 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/3b9cb49981844da2b5539fbec55d1c13 2020-11-17 03:45:37,461 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:45:37,479 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/3b9cb49981844da2b5539fbec55d1c13, entries=371657, sequenceid=259372025, filesize=6.9 M 2020-11-17 03:45:37,480 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.15 MB/106063378, heapSize ~128.00 MB/134218328, currentSize=36.22 KB/37085 for 06f7ca9534710e37d90d9f8345864259 in 804ms, sequenceid=259372025, compaction requested=true 2020-11-17 03:45:37,481 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 03:45:37,481 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/7861dd70448f42e39ce51d0b206f75f1, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/72fdc106413a4668bf9746873e2aab38, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/3b9cb49981844da2b5539fbec55d1c13] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=66.9 M 2020-11-17 03:45:37,514 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:45:37,529 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:45:37,555 INFO [Thread-1339] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:45:40,408 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.17 MB heapSize=128.08 MB 2020-11-17 03:45:40,457 INFO [Thread-1341] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:45:41,861 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.17 MB at sequenceid=290903812 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/340b70b9c1a54923ae32c45eaa11dbb3 2020-11-17 03:45:41,870 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:45:41,886 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/340b70b9c1a54923ae32c45eaa11dbb3, entries=554134, sequenceid=290903812, filesize=8.7 M 2020-11-17 03:45:41,888 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~88.17 MB/92452168, heapSize ~128.08 MB/134306712, currentSize=69.60 KB/71275 for bdaa31a63b7f3d06d9909b7560333e79 in 1480ms, sequenceid=290903812, compaction requested=true 2020-11-17 03:45:41,889 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-17 03:45:41,889 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/0d76d24b3bab47f2aa9d21f50a4ba84d, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/849815c9112744eeaaa6d439ba6acb50, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/340b70b9c1a54923ae32c45eaa11dbb3] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp, totalSize=101.5 M 2020-11-17 03:45:41,932 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:45:41,939 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:45:41,965 INFO [Thread-1343] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:45:56,882 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:45:57,497 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:45:58,073 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584656513 with entries=38149, filesize=128.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584758049 2020-11-17 03:45:58,074 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582813231 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605582813231 2020-11-17 03:45:58,086 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582924224 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605582924224 2020-11-17 03:45:58,092 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605582999658 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605582999658 2020-11-17 03:45:58,099 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583105430 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605583105430 2020-11-17 03:45:58,104 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583188600 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605583188600 2020-11-17 03:45:58,108 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583301115 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605583301115 2020-11-17 03:45:58,113 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583387725 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605583387725 2020-11-17 03:45:58,117 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583489933 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605583489933 2020-11-17 03:45:58,122 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583587147 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605583587147 2020-11-17 03:45:58,127 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583677946 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605583677946 2020-11-17 03:46:00,396 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:46:03,361 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#284 average throughput is 34.74 MB/second, slept 15 time(s) and total slept time is 15330 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 03:46:03,471 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 649b4ac81e604b518b948cea37d413fb(size=67.2 M), total size for store is 55.3 G. This selection was in queue for 0sec, and took 25sec to execute. 2020-11-17 03:46:03,471 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605584737480; duration=25sec 2020-11-17 03:46:04,470 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:46:05,720 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:46:08,827 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: bdaa31a63b7f3d06d9909b7560333e79#e#compaction#286 average throughput is 36.73 MB/second, slept 19 time(s) and total slept time is 13375 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 03:46:08,883 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of bdaa31a63b7f3d06d9909b7560333e79 into 646a21324266408ab1c324f48dd2a24d(size=100.7 M), total size for store is 31.8 G. This selection was in queue for 0sec, and took 26sec to execute. 2020-11-17 03:46:08,883 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., storeName=e, priority=11, startTime=1605584741888; duration=26sec 2020-11-17 03:46:20,193 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:46:34,415 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:46:42,157 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:46:47,987 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:46:56,371 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.05 MB heapSize=128.03 MB 2020-11-17 03:46:56,404 INFO [Thread-1349] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:46:57,074 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.05 MB at sequenceid=251695202 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/e574ef2e25d34e3780c71fabfdc34537 2020-11-17 03:46:57,080 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:46:57,096 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/e574ef2e25d34e3780c71fabfdc34537, entries=314264, sequenceid=251695202, filesize=6.4 M 2020-11-17 03:46:57,097 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.05 MB/111204669, heapSize ~128.03 MB/134253200, currentSize=50.16 KB/51363 for 82421ce8c5c8b48c573baee83caa8305 in 726ms, sequenceid=251695202, compaction requested=true 2020-11-17 03:46:57,098 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 03:46:57,098 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/96c4794418d34a24893e482158d048ed, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d7f2734f0a4a46c886f57e80337c5741, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/e574ef2e25d34e3780c71fabfdc34537] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=19.1 M 2020-11-17 03:46:57,123 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:46:57,125 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:46:57,224 INFO [Thread-1351] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:47:02,524 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#288 average throughput is 59.10 MB/second, slept 6 time(s) and total slept time is 2659 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 03:47:02,587 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 0749077180654ab290097cc6158ad46f(size=19.1 M), total size for store is 11.0 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 03:47:02,588 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605584817097; duration=5sec 2020-11-17 03:47:09,252 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:47:25,790 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:47:38,734 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 145039 ns. 2020-11-17 03:47:38,734 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 61904 ns. 2020-11-17 03:47:38,734 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 56593 ns. 2020-11-17 03:47:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 69101 ns. 2020-11-17 03:47:49,587 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:47:50,415 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584758049 with entries=42031, filesize=128.27 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584870367 2020-11-17 03:47:54,875 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:48:01,294 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.14 MB heapSize=128.00 MB 2020-11-17 03:48:01,332 INFO [Thread-1356] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:48:02,221 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.14 MB at sequenceid=262723359 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/f5a6d54b698745aa84361b4fb883d446 2020-11-17 03:48:02,229 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:48:02,255 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f5a6d54b698745aa84361b4fb883d446, entries=373254, sequenceid=262723359, filesize=8.1 M 2020-11-17 03:48:02,257 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.14 MB/106048843, heapSize ~128.00 MB/134222320, currentSize=260.18 KB/266426 for e5dc5c620be19b02666a19c280ff582e in 964ms, sequenceid=262723359, compaction requested=true 2020-11-17 03:48:11,250 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:49:03,513 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584870367 with entries=27707, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584943489 2020-11-17 03:49:18,764 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 24399 ns. 2020-11-17 03:49:18,765 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 40186 ns. 2020-11-17 03:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=176.46 MB, max=1.83 GB, blockCount=44084, accesses=12993264, hits=11252247, hitRatio=86.60%, , cachingAccesses=11303291, cachingHits=11249220, cachingHitsRatio=99.52%, evictions=1559, evicted=0, evictedPerRun=0.0 2020-11-17 03:49:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 135625 ns. 2020-11-17 03:50:15,209 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584943489 with entries=27752, filesize=128.32 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585015174 2020-11-17 03:50:26,100 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.84 MB heapSize=128.15 MB 2020-11-17 03:50:26,132 INFO [Thread-1364] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:50:27,236 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.84 MB at sequenceid=272682403 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/061b048d805d4d7bba4f848576396b24 2020-11-17 03:50:27,242 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:50:27,263 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/061b048d805d4d7bba4f848576396b24, entries=278884, sequenceid=272682403, filesize=8.5 M 2020-11-17 03:50:27,265 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~107.84 MB/113077100, heapSize ~128.15 MB/134371480, currentSize=28.03 KB/28705 for fc93f6dcc564341dcc983bf30971f73d in 1166ms, sequenceid=272682403, compaction requested=true 2020-11-17 03:50:39,208 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:50:49,242 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.57 MB heapSize=128.28 MB 2020-11-17 03:50:49,276 INFO [Thread-1367] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:50:49,856 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.57 MB at sequenceid=251702469 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/db5f8319dd9a4cc0b1783fc049f21120 2020-11-17 03:50:49,862 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:50:49,891 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/db5f8319dd9a4cc0b1783fc049f21120, entries=310521, sequenceid=251702469, filesize=6.3 M 2020-11-17 03:50:49,899 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.57 MB/111745207, heapSize ~128.28 MB/134514408, currentSize=482.63 KB/494214 for 82421ce8c5c8b48c573baee83caa8305 in 657ms, sequenceid=251702469, compaction requested=true 2020-11-17 03:51:36,299 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.23 MB heapSize=128.19 MB 2020-11-17 03:51:36,330 INFO [Thread-1370] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:51:36,882 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585015174 with entries=30309, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585096836 2020-11-17 03:51:37,156 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.23 MB at sequenceid=262734512 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/ddbb3c54e87142429bbad3590626ae4a 2020-11-17 03:51:37,169 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:51:37,188 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ddbb3c54e87142429bbad3590626ae4a, entries=416230, sequenceid=262734512, filesize=8.1 M 2020-11-17 03:51:37,190 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.23 MB/103004661, heapSize ~128.19 MB/134414664, currentSize=2.31 MB/2424160 for e5dc5c620be19b02666a19c280ff582e in 892ms, sequenceid=262734512, compaction requested=true 2020-11-17 03:51:37,190 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 03:51:37,191 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/4638fdd37efb4dc6a7d80dc00f7b6f35, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f5a6d54b698745aa84361b4fb883d446, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ddbb3c54e87142429bbad3590626ae4a] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=94.2 M 2020-11-17 03:51:37,217 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:51:37,219 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:51:37,251 INFO [Thread-1372] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:51:39,316 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:51:55,385 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#293 average throughput is 61.06 MB/second, slept 20 time(s) and total slept time is 6209 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 03:51:55,471 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 2a7699d75f16466eb826dfee3c03b985(size=95.7 M), total size for store is 48.8 G. This selection was in queue for 0sec, and took 18sec to execute. 2020-11-17 03:51:55,471 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605585097190; duration=18sec 2020-11-17 03:52:32,178 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585096836 with entries=20709, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585152114 2020-11-17 03:52:39,203 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 47788 ns. 2020-11-17 03:53:12,603 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.26 MB heapSize=128.00 MB 2020-11-17 03:53:12,647 INFO [Thread-1379] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:53:13,622 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.26 MB at sequenceid=287043594 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/68f4343ee7fc489eb0e94c7090e8124e 2020-11-17 03:53:13,628 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:53:13,645 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/68f4343ee7fc489eb0e94c7090e8124e, entries=527507, sequenceid=287043594, filesize=9.0 M 2020-11-17 03:53:13,647 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.26 MB/94641784, heapSize ~128.00 MB/134218304, currentSize=241.21 KB/247003 for 5e5f05205d7e40b0141cd977dd009ba1 in 1044ms, sequenceid=287043594, compaction requested=true 2020-11-17 03:53:13,648 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 03:53:13,648 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/6afe171935cd46019c36621c834f0645, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/3d26705a67a94019ae10daf8b6ca9ed0, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/68f4343ee7fc489eb0e94c7090e8124e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=46.0 M 2020-11-17 03:53:13,683 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:53:13,687 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:53:13,713 INFO [Thread-1381] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:53:18,054 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:53:20,931 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:53:21,264 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#295 average throughput is 59.06 MB/second, slept 7 time(s) and total slept time is 1957 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 03:53:21,355 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into a11e6288393d4dc3bd44c102c195664e(size=46.5 M), total size for store is 24.2 G. This selection was in queue for 0sec, and took 7sec to execute. 2020-11-17 03:53:21,355 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=11, startTime=1605585193647; duration=7sec 2020-11-17 03:53:38,771 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 59364 ns. 2020-11-17 03:53:38,772 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 187409 ns. 2020-11-17 03:53:38,772 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 61482 ns. 2020-11-17 03:54:28,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 13948 ns. 2020-11-17 03:54:28,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 50826 ns. 2020-11-17 03:54:28,764 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585152114 with entries=43016, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585268734 2020-11-17 03:54:28,764 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583785111 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605583785111 2020-11-17 03:54:28,773 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583864989 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605583864989 2020-11-17 03:54:28,781 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605583980608 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605583980608 2020-11-17 03:54:28,786 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584053405 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605584053405 2020-11-17 03:54:28,791 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584163377 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605584163377 2020-11-17 03:54:28,796 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584232468 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605584232468 2020-11-17 03:54:28,801 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584342770 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605584342770 2020-11-17 03:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=177.17 MB, max=1.83 GB, blockCount=44060, accesses=13021293, hits=11254938, hitRatio=86.43%, , cachingAccesses=11305967, cachingHits=11251889, cachingHitsRatio=99.52%, evictions=1589, evicted=0, evictedPerRun=0.0 2020-11-17 03:55:00,579 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.71 MB heapSize=128.21 MB 2020-11-17 03:55:00,621 INFO [Thread-1388] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:55:01,525 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.71 MB at sequenceid=262744612 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/192b9eb66a044c36afab99d661d8acde 2020-11-17 03:55:01,534 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:55:01,554 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/192b9eb66a044c36afab99d661d8acde, entries=367490, sequenceid=262744612, filesize=8.2 M 2020-11-17 03:55:01,557 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.71 MB/106654021, heapSize ~128.21 MB/134434952, currentSize=739.12 KB/756858 for e5dc5c620be19b02666a19c280ff582e in 979ms, sequenceid=262744612, compaction requested=true 2020-11-17 03:55:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 69585 ns. 2020-11-17 03:55:57,065 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:56:04,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:56:07,984 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:56:08,084 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.85 MB heapSize=128.00 MB 2020-11-17 03:56:08,162 INFO [Thread-1393] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:56:09,430 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.85 MB at sequenceid=261247456 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/ea8458dee2194fe38b0d679df7be3fd3 2020-11-17 03:56:09,470 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/ea8458dee2194fe38b0d679df7be3fd3, entries=571787, sequenceid=261247456, filesize=8.9 M 2020-11-17 03:56:09,473 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.85 MB/90017213, heapSize ~128.00 MB/134220912, currentSize=49.15 KB/50328 for 73aae5715542e1a5370f7540d2117abc in 1390ms, sequenceid=261247456, compaction requested=true 2020-11-17 03:56:09,473 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 03:56:09,474 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/83924e9b4f9e4aa1848516bb502a50ea, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/71f01d6bb3594096a827294f16a75dfc, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/ea8458dee2194fe38b0d679df7be3fd3] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=44.9 M 2020-11-17 03:56:09,505 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:56:09,509 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:56:09,552 INFO [Thread-1395] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:56:10,077 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:56:19,150 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#298 average throughput is 43.18 MB/second, slept 1 time(s) and total slept time is 234 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 03:56:19,213 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into a4deacaa02fc40baa33c7f482bca2564(size=45.6 M), total size for store is 12.5 G. This selection was in queue for 0sec, and took 9sec to execute. 2020-11-17 03:56:19,213 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605585369473; duration=9sec 2020-11-17 03:56:19,572 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585268734 with entries=39793, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585379527 2020-11-17 03:56:19,573 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584416445 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605584416445 2020-11-17 03:56:19,583 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584524834 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605584524834 2020-11-17 03:56:20,476 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.84 MB heapSize=128.00 MB 2020-11-17 03:56:20,511 INFO [Thread-1397] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:56:21,313 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.84 MB at sequenceid=271556901 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/50d3d9b4ee2243a6a562e1c90ad42959 2020-11-17 03:56:21,345 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/50d3d9b4ee2243a6a562e1c90ad42959, entries=318353, sequenceid=271556901, filesize=8.7 M 2020-11-17 03:56:21,346 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.84 MB/109936057, heapSize ~128.00 MB/134219848, currentSize=56.43 KB/57785 for 0469ce862509756add4a8cdb00bc465c in 870ms, sequenceid=271556901, compaction requested=true 2020-11-17 03:56:21,347 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 03:56:21,348 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/7d95767f650e44e6981473b78766033d, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/4bc268c95caa493b92cf0db92e20b3e1, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/50d3d9b4ee2243a6a562e1c90ad42959] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=62.5 M 2020-11-17 03:56:21,430 INFO [Thread-1399] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:56:23,742 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:56:29,009 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.86 MB heapSize=128.00 MB 2020-11-17 03:56:29,071 INFO [Thread-1401] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:56:30,390 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.86 MB at sequenceid=259406264 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/b51aedb509f7417ab0b1110668d5685e 2020-11-17 03:56:30,399 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:56:30,445 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/b51aedb509f7417ab0b1110668d5685e, entries=373451, sequenceid=259406264, filesize=7.1 M 2020-11-17 03:56:30,447 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.86 MB/105763211, heapSize ~128.00 MB/134219200, currentSize=31.97 KB/32736 for 06f7ca9534710e37d90d9f8345864259 in 1438ms, sequenceid=259406264, compaction requested=true 2020-11-17 03:56:36,097 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:56:36,478 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#300 average throughput is 48.55 MB/second, slept 10 time(s) and total slept time is 2729 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 03:56:36,564 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into e1051e0d35e84c52aa3b61e33456ea9a(size=63.3 M), total size for store is 42.7 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 03:56:36,564 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=11, startTime=1605585381347; duration=15sec 2020-11-17 03:56:47,988 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:56:59,296 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:57:06,186 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:57:56,155 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:58:16,954 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585379527 with entries=43460, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585496919 2020-11-17 03:58:16,955 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584567349 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605584567349 2020-11-17 03:58:18,988 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.04 MB heapSize=128.05 MB 2020-11-17 03:58:19,030 INFO [Thread-1410] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:58:19,753 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.04 MB at sequenceid=262754484 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/385f45fbd37449db81cba1d3143b0c73 2020-11-17 03:58:19,759 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:58:19,778 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/385f45fbd37449db81cba1d3143b0c73, entries=346929, sequenceid=262754484, filesize=8.2 M 2020-11-17 03:58:19,779 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.04 MB/108048727, heapSize ~128.05 MB/134275072, currentSize=535.49 KB/548341 for e5dc5c620be19b02666a19c280ff582e in 791ms, sequenceid=262754484, compaction requested=true 2020-11-17 03:58:19,781 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 03:58:19,781 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2a7699d75f16466eb826dfee3c03b985, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/192b9eb66a044c36afab99d661d8acde, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/385f45fbd37449db81cba1d3143b0c73] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=112.1 M 2020-11-17 03:58:19,823 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:58:19,825 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:58:19,850 INFO [Thread-1412] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:58:22,155 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:58:35,803 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:58:38,760 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 44424 ns. 2020-11-17 03:58:42,754 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#303 average throughput is 57.29 MB/second, slept 22 time(s) and total slept time is 7226 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 03:58:42,859 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 13465277ab2247d1a5206bda6b7a3222(size=113.5 M), total size for store is 48.8 G. This selection was in queue for 0sec, and took 23sec to execute. 2020-11-17 03:58:42,859 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605585499780; duration=23sec 2020-11-17 03:59:13,594 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:59:28,733 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18313 ns. 2020-11-17 03:59:28,733 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 49739 ns. 2020-11-17 03:59:30,513 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.61 MB heapSize=128.00 MB 2020-11-17 03:59:30,548 INFO [Thread-1417] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:59:31,291 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.61 MB at sequenceid=251717482 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/fe601d00d69044489415bf5cc5c4ada8 2020-11-17 03:59:31,298 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:59:31,316 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/fe601d00d69044489415bf5cc5c4ada8, entries=319495, sequenceid=251717482, filesize=6.5 M 2020-11-17 03:59:31,317 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.61 MB/110744959, heapSize ~128.00 MB/134218408, currentSize=14.42 KB/14763 for 82421ce8c5c8b48c573baee83caa8305 in 804ms, sequenceid=251717482, compaction requested=true 2020-11-17 03:59:31,318 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 03:59:31,318 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0749077180654ab290097cc6158ad46f, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/db5f8319dd9a4cc0b1783fc049f21120, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/fe601d00d69044489415bf5cc5c4ada8] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=31.9 M 2020-11-17 03:59:31,348 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:59:31,355 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:59:31,378 INFO [Thread-1419] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 03:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=177.03 MB, max=1.83 GB, blockCount=44057, accesses=13325324, hits=11514060, hitRatio=86.41%, , cachingAccesses=11565104, cachingHits=11510968, cachingHitsRatio=99.53%, evictions=1619, evicted=0, evictedPerRun=0.0 2020-11-17 03:59:39,503 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 243948 ns. 2020-11-17 03:59:39,503 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 72504 ns. 2020-11-17 03:59:39,503 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 70582 ns. 2020-11-17 03:59:39,786 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#305 average throughput is 63.02 MB/second, slept 10 time(s) and total slept time is 3145 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 03:59:39,860 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into dd69a4d288e245d589bbfe7d4e4f42f5(size=31.8 M), total size for store is 11.1 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 03:59:39,860 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605585571317; duration=8sec 2020-11-17 04:00:39,287 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 66142 ns. 2020-11-17 04:00:49,192 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585496919 with entries=55993, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585649159 2020-11-17 04:02:00,431 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:02:05,944 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:02:57,351 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585649159 with entries=47323, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585777320 2020-11-17 04:03:04,942 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.19 MB heapSize=128.01 MB 2020-11-17 04:03:04,970 INFO [Thread-1430] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:03:05,700 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.19 MB at sequenceid=262768863 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fec03712115f480a8d3f484b77bb0cd9 2020-11-17 04:03:05,709 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:03:05,841 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/fec03712115f480a8d3f484b77bb0cd9, entries=372999, sequenceid=262768863, filesize=8.2 M 2020-11-17 04:03:05,846 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.19 MB/106100857, heapSize ~128.01 MB/134223112, currentSize=114.51 KB/117255 for e5dc5c620be19b02666a19c280ff582e in 905ms, sequenceid=262768863, compaction requested=true 2020-11-17 04:03:18,052 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:03:22,293 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:03:38,769 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 47722 ns. 2020-11-17 04:03:41,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:03:51,838 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:04:01,145 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:04:13,933 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:04:27,206 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:04:28,742 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 16360 ns. 2020-11-17 04:04:28,742 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 45083 ns. 2020-11-17 04:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=174.13 MB, max=1.83 GB, blockCount=44139, accesses=13501322, hits=11686209, hitRatio=86.56%, , cachingAccesses=11737360, cachingHits=11683117, cachingHitsRatio=99.54%, evictions=1649, evicted=0, evictedPerRun=0.0 2020-11-17 04:04:39,528 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585777320 with entries=37315, filesize=128.30 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585879478 2020-11-17 04:04:46,356 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:04:49,407 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:05:09,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:05:18,147 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:05:26,805 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:05:38,961 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 276293 ns. 2020-11-17 04:05:38,961 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 88034 ns. 2020-11-17 04:05:38,961 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 114782 ns. 2020-11-17 04:05:45,823 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:05:54,853 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:06:05,503 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:06:09,694 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.09 MB heapSize=128.25 MB 2020-11-17 04:06:09,761 INFO [Thread-1440] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:06:10,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:06:10,434 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.09 MB at sequenceid=272710967 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/edc9e1d68c0241eca0bba06e879fae01 2020-11-17 04:06:10,469 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/edc9e1d68c0241eca0bba06e879fae01, entries=276355, sequenceid=272710967, filesize=8.5 M 2020-11-17 04:06:10,472 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.09 MB/113344235, heapSize ~128.25 MB/134474720, currentSize=23.28 KB/23836 for fc93f6dcc564341dcc983bf30971f73d in 779ms, sequenceid=272710967, compaction requested=true 2020-11-17 04:06:10,473 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 04:06:10,473 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/d49077802f5d4db7ad896a2b67f75b42, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/061b048d805d4d7bba4f848576396b24, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/edc9e1d68c0241eca0bba06e879fae01] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=60.7 M 2020-11-17 04:06:10,502 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:06:10,506 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:06:10,525 INFO [Thread-1442] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:06:10,542 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:06:18,115 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:06:23,631 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#308 average throughput is 57.58 MB/second, slept 15 time(s) and total slept time is 6486 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 04:06:23,715 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into aff66dbaa8094de9a703a54a4c24bbd5(size=61.3 M), total size for store is 33.3 G. This selection was in queue for 0sec, and took 13sec to execute. 2020-11-17 04:06:23,715 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605585970472; duration=13sec 2020-11-17 04:06:31,470 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:06:38,019 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:06:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 55450 ns. 2020-11-17 04:06:49,994 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:06:50,661 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.20 MB heapSize=128.10 MB 2020-11-17 04:06:50,699 INFO [Thread-1447] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:06:51,579 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.20 MB at sequenceid=262780177 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/b38c87d484e841c0ac1663b855c97ec7 2020-11-17 04:06:51,602 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b38c87d484e841c0ac1663b855c97ec7, entries=402235, sequenceid=262780177, filesize=8.1 M 2020-11-17 04:06:51,604 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~99.20 MB/104014082, heapSize ~128.10 MB/134317552, currentSize=545.26 KB/558348 for e5dc5c620be19b02666a19c280ff582e in 943ms, sequenceid=262780177, compaction requested=true 2020-11-17 04:07:04,955 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585879478 with entries=52822, filesize=128.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586024932 2020-11-17 04:07:07,460 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:07:10,848 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:08:39,482 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 66306 ns. 2020-11-17 04:09:29,266 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 38979 ns. 2020-11-17 04:09:29,266 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 19201 ns. 2020-11-17 04:09:30,118 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586024932 with entries=53578, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586170079 2020-11-17 04:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.97 MB, max=1.83 GB, blockCount=44215, accesses=14017398, hits=12190006, hitRatio=86.96%, , cachingAccesses=12241241, cachingHits=12186910, cachingHitsRatio=99.56%, evictions=1679, evicted=0, evictedPerRun=0.0 2020-11-17 04:09:53,604 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.01 MB heapSize=128.00 MB 2020-11-17 04:09:53,669 INFO [Thread-1456] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:09:54,458 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.01 MB at sequenceid=271596741 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/5aa083855e8c422392a73aeaaf43d7c0 2020-11-17 04:09:54,464 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:09:54,481 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/5aa083855e8c422392a73aeaaf43d7c0, entries=315991, sequenceid=271596741, filesize=8.7 M 2020-11-17 04:09:54,483 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.01 MB/110106486, heapSize ~128.00 MB/134218168, currentSize=31.68 KB/32442 for 0469ce862509756add4a8cdb00bc465c in 879ms, sequenceid=271596741, compaction requested=true 2020-11-17 04:10:21,253 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.94 MB heapSize=128.05 MB 2020-11-17 04:10:21,297 INFO [Thread-1459] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:10:22,035 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.94 MB at sequenceid=262790841 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/edc13111f3ef4b7ab1aa6465db9233f9 2020-11-17 04:10:22,042 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:10:22,061 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/edc13111f3ef4b7ab1aa6465db9233f9, entries=348353, sequenceid=262790841, filesize=8.2 M 2020-11-17 04:10:22,062 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.94 MB/107938058, heapSize ~128.05 MB/134270888, currentSize=905.71 KB/927446 for e5dc5c620be19b02666a19c280ff582e in 809ms, sequenceid=262790841, compaction requested=true 2020-11-17 04:10:22,063 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 04:10:22,063 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/fec03712115f480a8d3f484b77bb0cd9, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b38c87d484e841c0ac1663b855c97ec7, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/edc13111f3ef4b7ab1aa6465db9233f9] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.5 M 2020-11-17 04:10:22,094 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:10:22,095 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:10:22,122 INFO [Thread-1461] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:10:22,728 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:10:27,016 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#312 average throughput is 61.72 MB/second, slept 6 time(s) and total slept time is 1785 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 04:10:27,098 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 88caf604260d4ca7960cec94cd843862(size=25.4 M), total size for store is 48.9 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 04:10:27,099 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605586222063; duration=5sec 2020-11-17 04:11:18,331 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586170079 with entries=39896, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586278302 2020-11-17 04:11:19,860 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:11:26,558 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:11:33,016 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:11:39,119 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 110630 ns. 2020-11-17 04:11:39,119 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 328589 ns. 2020-11-17 04:11:39,119 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 48517 ns. 2020-11-17 04:11:39,119 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 99847 ns. 2020-11-17 04:11:47,773 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.34 MB heapSize=128.00 MB 2020-11-17 04:11:47,803 INFO [Thread-1467] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:11:48,930 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.34 MB at sequenceid=261296226 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/a1318e36d92c40d8b36f202bf9761256 2020-11-17 04:11:48,940 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:11:48,971 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/a1318e36d92c40d8b36f202bf9761256, entries=576027, sequenceid=261296226, filesize=8.9 M 2020-11-17 04:11:48,973 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.34 MB/89484390, heapSize ~128.00 MB/134218128, currentSize=34.48 KB/35305 for 73aae5715542e1a5370f7540d2117abc in 1200ms, sequenceid=261296226, compaction requested=true 2020-11-17 04:12:16,116 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:13:16,950 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:13:21,078 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:13:27,224 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586278302 with entries=47167, filesize=128.20 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586407181 2020-11-17 04:13:32,995 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:13:42,011 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:13:59,379 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:14:12,059 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.69 MB, max=1.83 GB, blockCount=44220, accesses=14280830, hits=12448499, hitRatio=87.17%, , cachingAccesses=12499744, cachingHits=12445390, cachingHitsRatio=99.57%, evictions=1709, evicted=0, evictedPerRun=0.0 2020-11-17 04:14:38,824 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 53515 ns. 2020-11-17 04:14:38,824 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 13436 ns. 2020-11-17 04:14:38,825 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 45061 ns. 2020-11-17 04:15:02,343 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.30 MB heapSize=128.00 MB 2020-11-17 04:15:02,379 INFO [Thread-1475] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:15:03,248 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.30 MB at sequenceid=262804923 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/c4a29ed0af354f4188a97ba2f9b8d9e4 2020-11-17 04:15:03,255 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:15:03,275 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c4a29ed0af354f4188a97ba2f9b8d9e4, entries=356992, sequenceid=262804923, filesize=8.2 M 2020-11-17 04:15:03,277 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.30 MB/107269722, heapSize ~128.00 MB/134217944, currentSize=295.74 KB/302840 for e5dc5c620be19b02666a19c280ff582e in 934ms, sequenceid=262804923, compaction requested=true 2020-11-17 04:15:10,361 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:15:35,020 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586407181 with entries=47411, filesize=128.33 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586534992 2020-11-17 04:16:09,638 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:16:42,022 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586534992 with entries=24881, filesize=128.08 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586601990 2020-11-17 04:17:38,789 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 65070 ns. 2020-11-17 04:17:38,789 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 294885 ns. 2020-11-17 04:17:38,790 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 105613 ns. 2020-11-17 04:17:38,790 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 136054 ns. 2020-11-17 04:18:01,219 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:13,210 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.93 MB heapSize=128.19 MB 2020-11-17 04:18:13,222 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=97.18 MB heapSize=128.10 MB 2020-11-17 04:18:13,267 INFO [Thread-1487] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:13,267 INFO [Thread-1488] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:14,096 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.18 MB at sequenceid=259473748 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/04e0c29e29744173b80332a65824590e 2020-11-17 04:18:14,105 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:14,127 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/04e0c29e29744173b80332a65824590e, entries=422835, sequenceid=259473748, filesize=8.2 M 2020-11-17 04:18:14,129 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~97.18 MB/101905074, heapSize ~128.10 MB/134323960, currentSize=44.63 KB/45704 for 06f7ca9534710e37d90d9f8345864259 in 907ms, sequenceid=259473748, compaction requested=true 2020-11-17 04:18:14,131 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 04:18:14,131 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/649b4ac81e604b518b948cea37d413fb, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/b51aedb509f7417ab0b1110668d5685e, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/04e0c29e29744173b80332a65824590e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=82.5 M 2020-11-17 04:18:14,176 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:14,183 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:14,215 INFO [Thread-1491] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:14,377 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.93 MB at sequenceid=287124973 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/d9110d02c75e48699fce82c20bc58e0b 2020-11-17 04:18:14,384 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:14,400 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/d9110d02c75e48699fce82c20bc58e0b, entries=520073, sequenceid=287124973, filesize=9.3 M 2020-11-17 04:18:14,402 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.93 MB/95347517, heapSize ~128.19 MB/134416816, currentSize=232.23 KB/237799 for 5e5f05205d7e40b0141cd977dd009ba1 in 1192ms, sequenceid=287124973, compaction requested=true 2020-11-17 04:18:32,584 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#317 average throughput is 59.46 MB/second, slept 17 time(s) and total slept time is 5893 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 04:18:32,672 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 144fb4fb493f459990ebff2b637b25d9(size=82.9 M), total size for store is 55.4 G. This selection was in queue for 0sec, and took 18sec to execute. 2020-11-17 04:18:32,672 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605586694130; duration=18sec 2020-11-17 04:18:37,401 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=104.17 MB heapSize=128.25 MB 2020-11-17 04:18:37,434 INFO [Thread-1494] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:38,127 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.17 MB at sequenceid=251750374 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/f2c4b103e83d45cd87ed9268e9917a7a 2020-11-17 04:18:38,132 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:38,150 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/f2c4b103e83d45cd87ed9268e9917a7a, entries=342203, sequenceid=251750374, filesize=7.1 M 2020-11-17 04:18:38,152 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.17 MB/109226949, heapSize ~128.25 MB/134476272, currentSize=32.87 KB/33663 for 82421ce8c5c8b48c573baee83caa8305 in 751ms, sequenceid=251750374, compaction requested=true 2020-11-17 04:18:41,222 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586601990 with entries=44012, filesize=128.14 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586721199 2020-11-17 04:18:42,459 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.02 MB heapSize=128.17 MB 2020-11-17 04:18:42,486 INFO [Thread-1496] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:43,416 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.02 MB at sequenceid=262816106 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/c612646f938049baac3289041d7bb859 2020-11-17 04:18:43,423 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:43,443 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c612646f938049baac3289041d7bb859, entries=405599, sequenceid=262816106, filesize=8.0 M 2020-11-17 04:18:43,444 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.02 MB/103834506, heapSize ~128.17 MB/134395152, currentSize=794.75 KB/813828 for e5dc5c620be19b02666a19c280ff582e in 985ms, sequenceid=262816106, compaction requested=true 2020-11-17 04:18:43,445 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 04:18:43,445 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/88caf604260d4ca7960cec94cd843862, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c4a29ed0af354f4188a97ba2f9b8d9e4, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c612646f938049baac3289041d7bb859] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.7 M 2020-11-17 04:18:43,487 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:43,489 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:43,517 INFO [Thread-1498] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:45,725 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:18:52,574 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#320 average throughput is 55.59 MB/second, slept 9 time(s) and total slept time is 2354 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 04:18:52,656 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 06cf75148ba64419b6f905386adc344e(size=43.1 M), total size for store is 48.9 G. This selection was in queue for 0sec, and took 9sec to execute. 2020-11-17 04:18:52,656 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605586723444; duration=9sec 2020-11-17 04:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.53 MB, max=1.83 GB, blockCount=44232, accesses=14312881, hits=12454583, hitRatio=87.02%, , cachingAccesses=12505802, cachingHits=12451436, cachingHitsRatio=99.57%, evictions=1739, evicted=0, evictedPerRun=0.0 2020-11-17 04:19:39,318 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 48391 ns. 2020-11-17 04:19:39,319 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 7341 ns. 2020-11-17 04:19:39,319 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 34502 ns. 2020-11-17 04:19:40,415 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.62 MB heapSize=128.01 MB 2020-11-17 04:19:40,452 INFO [Thread-1502] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:19:41,682 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.62 MB at sequenceid=291010334 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/5ca0cbe824c649b38ed115679eba0157 2020-11-17 04:19:41,688 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:19:41,704 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/5ca0cbe824c649b38ed115679eba0157, entries=547720, sequenceid=291010334, filesize=9.1 M 2020-11-17 04:19:41,705 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~88.62 MB/92926566, heapSize ~128.01 MB/134228568, currentSize=62.17 KB/63661 for bdaa31a63b7f3d06d9909b7560333e79 in 1290ms, sequenceid=291010334, compaction requested=true 2020-11-17 04:19:51,195 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586721199 with entries=25863, filesize=128.17 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586791158 2020-11-17 04:19:51,196 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584656513 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605584656513 2020-11-17 04:19:51,216 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584758049 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605584758049 2020-11-17 04:19:51,226 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584870367 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605584870367 2020-11-17 04:19:51,233 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605584943489 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605584943489 2020-11-17 04:19:51,241 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585015174 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605585015174 2020-11-17 04:19:51,247 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585096836 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605585096836 2020-11-17 04:19:51,254 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585152114 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605585152114 2020-11-17 04:19:51,259 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585268734 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605585268734 2020-11-17 04:19:51,263 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585379527 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605585379527 2020-11-17 04:19:51,269 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585496919 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605585496919 2020-11-17 04:19:51,274 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585649159 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605585649159 2020-11-17 04:19:51,283 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585777320 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605585777320 2020-11-17 04:19:52,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:21:43,546 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.31 MB heapSize=128.40 MB 2020-11-17 04:21:43,576 INFO [Thread-1509] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:21:44,248 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.31 MB at sequenceid=272739456 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/509898668d99445ea11b1293ba3e313b 2020-11-17 04:21:44,255 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:21:44,273 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/509898668d99445ea11b1293ba3e313b, entries=275479, sequenceid=272739456, filesize=8.5 M 2020-11-17 04:21:44,274 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.31 MB/113571049, heapSize ~128.40 MB/134633288, currentSize=356.76 KB/365326 for fc93f6dcc564341dcc983bf30971f73d in 728ms, sequenceid=272739456, compaction requested=true 2020-11-17 04:21:50,949 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586791158 with entries=44189, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586910927 2020-11-17 04:21:50,949 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605585879478 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605585879478 2020-11-17 04:21:50,966 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586024932 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605586024932 2020-11-17 04:22:26,241 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.40 MB heapSize=128.49 MB 2020-11-17 04:22:26,282 INFO [Thread-1512] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:22:27,006 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.40 MB at sequenceid=262827439 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/d666401acf9948d9b02a3c99cb703beb 2020-11-17 04:22:27,012 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:22:27,031 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d666401acf9948d9b02a3c99cb703beb, entries=376020, sequenceid=262827439, filesize=8.3 M 2020-11-17 04:22:27,032 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.40 MB/106326269, heapSize ~128.49 MB/134730424, currentSize=699.03 KB/715811 for e5dc5c620be19b02666a19c280ff582e in 791ms, sequenceid=262827439, compaction requested=true 2020-11-17 04:22:38,600 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:22:38,867 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 49414 ns. 2020-11-17 04:23:11,260 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586910927 with entries=29475, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586991209 2020-11-17 04:23:11,725 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:23:18,062 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:23:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 278074 ns. 2020-11-17 04:23:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 102160 ns. 2020-11-17 04:23:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 106930 ns. 2020-11-17 04:23:42,492 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.91 MB heapSize=128.06 MB 2020-11-17 04:23:42,543 INFO [Thread-1518] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:23:43,323 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.91 MB at sequenceid=271636978 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/ed20aa8e3d34435ca63975c26a89b4e8 2020-11-17 04:23:43,329 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:23:43,346 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/ed20aa8e3d34435ca63975c26a89b4e8, entries=318356, sequenceid=271636978, filesize=8.7 M 2020-11-17 04:23:43,348 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.91 MB/110004483, heapSize ~128.06 MB/134280888, currentSize=817.52 KB/837136 for 0469ce862509756add4a8cdb00bc465c in 856ms, sequenceid=271636978, compaction requested=true 2020-11-17 04:23:43,348 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 04:23:43,349 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/e1051e0d35e84c52aa3b61e33456ea9a, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/5aa083855e8c422392a73aeaaf43d7c0, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/ed20aa8e3d34435ca63975c26a89b4e8] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=80.7 M 2020-11-17 04:23:43,376 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:23:43,380 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:23:43,399 INFO [Thread-1520] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:23:45,110 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:23:57,107 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:23:58,150 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:23:58,765 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#325 average throughput is 60.91 MB/second, slept 15 time(s) and total slept time is 5744 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 04:23:58,828 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 5acc867713164d75b4faa28be29664a8(size=81.3 M), total size for store is 42.7 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 04:23:58,828 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=11, startTime=1605587023348; duration=15sec 2020-11-17 04:24:11,101 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.60 MB heapSize=128.15 MB 2020-11-17 04:24:11,146 INFO [Thread-1524] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:24:11,812 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.60 MB at sequenceid=251760183 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/fe5cbf086e6b459db935767267bbbb98 2020-11-17 04:24:11,820 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:24:11,839 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/fe5cbf086e6b459db935767267bbbb98, entries=308217, sequenceid=251760183, filesize=6.4 M 2020-11-17 04:24:11,840 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.60 MB/111772997, heapSize ~128.15 MB/134376704, currentSize=438.98 KB/449517 for 82421ce8c5c8b48c573baee83caa8305 in 739ms, sequenceid=251760183, compaction requested=true 2020-11-17 04:24:11,841 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 04:24:11,841 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/dd69a4d288e245d589bbfe7d4e4f42f5, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/f2c4b103e83d45cd87ed9268e9917a7a, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/fe5cbf086e6b459db935767267bbbb98] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=45.3 M 2020-11-17 04:24:11,887 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:24:11,895 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:24:11,927 INFO [Thread-1526] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:24:24,314 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#327 average throughput is 59.75 MB/second, slept 13 time(s) and total slept time is 4912 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 04:24:24,410 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 79329c4839da40d7b740c87a679fe708(size=45.2 M), total size for store is 11.1 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 04:24:24,410 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605587051840; duration=12sec 2020-11-17 04:24:33,327 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.64 MB, max=1.83 GB, blockCount=44199, accesses=14347714, hits=12462240, hitRatio=86.86%, , cachingAccesses=12513460, cachingHits=12459074, cachingHitsRatio=99.57%, evictions=1769, evicted=0, evictedPerRun=0.0 2020-11-17 04:24:49,421 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 9793 ns. 2020-11-17 04:24:49,421 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 50747 ns. 2020-11-17 04:25:00,257 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586991209 with entries=39861, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587100225 2020-11-17 04:25:00,257 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586170079 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605586170079 2020-11-17 04:25:39,257 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 42600 ns. 2020-11-17 04:25:53,535 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.39 MB heapSize=128.11 MB 2020-11-17 04:25:53,576 INFO [Thread-1534] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:25:54,301 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.39 MB at sequenceid=262837819 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/64d5f457c00143298d0f6082d033026a 2020-11-17 04:25:54,308 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:25:54,326 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/64d5f457c00143298d0f6082d033026a, entries=356628, sequenceid=262837819, filesize=8.2 M 2020-11-17 04:25:54,328 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.39 MB/107367067, heapSize ~128.11 MB/134336696, currentSize=551.10 KB/564324 for e5dc5c620be19b02666a19c280ff582e in 794ms, sequenceid=262837819, compaction requested=true 2020-11-17 04:25:54,329 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 04:25:54,329 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/06cf75148ba64419b6f905386adc344e, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d666401acf9948d9b02a3c99cb703beb, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/64d5f457c00143298d0f6082d033026a] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=59.6 M 2020-11-17 04:25:54,359 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:25:54,361 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:25:54,383 INFO [Thread-1536] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:26:05,928 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#329 average throughput is 61.28 MB/second, slept 14 time(s) and total slept time is 4280 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 04:26:06,002 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 7bc1fb1b75aa4e0da2f2c865abe458c7(size=60.8 M), total size for store is 48.9 G. This selection was in queue for 0sec, and took 11sec to execute. 2020-11-17 04:26:06,002 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605587154329; duration=11sec 2020-11-17 04:26:20,528 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:26:24,376 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587100225 with entries=31492, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587184342 2020-11-17 04:26:49,581 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:26:54,205 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:28:05,971 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587184342 with entries=36917, filesize=128.39 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587285909 2020-11-17 04:28:06,026 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:28:38,056 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.07 MB heapSize=128.00 MB 2020-11-17 04:28:38,129 INFO [Thread-1544] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:28:39,172 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.07 MB at sequenceid=261348591 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/7eafcd8ed5794b5197c4e87443bbd756 2020-11-17 04:28:39,179 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:28:39,200 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/7eafcd8ed5794b5197c4e87443bbd756, entries=585975, sequenceid=261348591, filesize=9.0 M 2020-11-17 04:28:39,201 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.07 MB/89198374, heapSize ~128.00 MB/134222768, currentSize=251.24 KB/257265 for 73aae5715542e1a5370f7540d2117abc in 1145ms, sequenceid=261348591, compaction requested=true 2020-11-17 04:28:39,203 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 04:28:39,203 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/a4deacaa02fc40baa33c7f482bca2564, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/a1318e36d92c40d8b36f202bf9761256, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/7eafcd8ed5794b5197c4e87443bbd756] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=63.5 M 2020-11-17 04:28:39,264 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:28:39,268 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:28:39,297 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 70915 ns. 2020-11-17 04:28:39,300 INFO [Thread-1546] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:28:47,464 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:28:49,493 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#331 average throughput is 56.48 MB/second, slept 9 time(s) and total slept time is 1198 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 04:28:49,571 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 62b6c407e04c470d8341c4158c4b06d5(size=64.0 M), total size for store is 12.5 G. This selection was in queue for 0sec, and took 10sec to execute. 2020-11-17 04:28:49,571 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605587319202; duration=10sec 2020-11-17 04:29:08,085 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:29:13,058 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:29:15,199 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587285909 with entries=26223, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587355155 2020-11-17 04:29:15,199 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586278302 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605586278302 2020-11-17 04:29:15,215 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586407181 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605586407181 2020-11-17 04:29:15,223 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586534992 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605586534992 2020-11-17 04:29:16,896 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:29:23,902 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.70 MB, max=1.83 GB, blockCount=44194, accesses=14622035, hits=12715524, hitRatio=86.96%, , cachingAccesses=12766741, cachingHits=12712332, cachingHitsRatio=99.57%, evictions=1799, evicted=0, evictedPerRun=0.0 2020-11-17 04:29:36,028 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 04:29:39,070 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 298101 ns. 2020-11-17 04:29:39,071 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 105053 ns. 2020-11-17 04:29:39,071 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 84717 ns. 2020-11-17 04:29:39,359 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.44 MB heapSize=128.18 MB 2020-11-17 04:29:39,390 INFO [Thread-1551] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:29:40,054 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.44 MB at sequenceid=251769775 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/8b23014d109549208372156b2442de3e 2020-11-17 04:29:40,061 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:29:40,084 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/8b23014d109549208372156b2442de3e, entries=310871, sequenceid=251769775, filesize=6.4 M 2020-11-17 04:29:40,086 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.44 MB/111605776, heapSize ~128.18 MB/134402936, currentSize=225.98 KB/231402 for 82421ce8c5c8b48c573baee83caa8305 in 727ms, sequenceid=251769775, compaction requested=true 2020-11-17 04:29:58,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 29295 ns. 2020-11-17 04:29:58,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 44376 ns. 2020-11-17 04:30:14,705 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.01 MB heapSize=128.42 MB 2020-11-17 04:30:14,763 INFO [Thread-1555] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:30:15,668 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.01 MB at sequenceid=262850974 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/6f48c7563c9b4e0ca57eb88dc130ffb7 2020-11-17 04:30:15,674 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:30:15,693 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6f48c7563c9b4e0ca57eb88dc130ffb7, entries=380777, sequenceid=262850974, filesize=8.1 M 2020-11-17 04:30:15,694 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.01 MB/105912966, heapSize ~128.42 MB/134655800, currentSize=331.81 KB/339774 for e5dc5c620be19b02666a19c280ff582e in 989ms, sequenceid=262850974, compaction requested=true 2020-11-17 04:30:15,935 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.64 MB heapSize=128.02 MB 2020-11-17 04:30:15,965 INFO [Thread-1557] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:30:16,801 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.64 MB at sequenceid=259511318 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/5acc03e5c6694c73b562b090db235d84 2020-11-17 04:30:16,827 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/5acc03e5c6694c73b562b090db235d84, entries=377876, sequenceid=259511318, filesize=7.2 M 2020-11-17 04:30:16,829 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.64 MB/105529397, heapSize ~128.02 MB/134243264, currentSize=64.88 KB/66433 for 06f7ca9534710e37d90d9f8345864259 in 894ms, sequenceid=259511318, compaction requested=true 2020-11-17 04:30:56,694 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587355155 with entries=37202, filesize=128.65 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587456643 2020-11-17 04:31:11,929 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:31:15,616 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:31:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 55323 ns. 2020-11-17 04:32:16,913 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587456643 with entries=29940, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587536873 2020-11-17 04:33:18,053 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:34:01,361 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587536873 with entries=38348, filesize=128.25 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587641315 2020-11-17 04:34:01,649 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.18 MB heapSize=128.16 MB 2020-11-17 04:34:01,679 INFO [Thread-1569] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:34:02,583 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.18 MB at sequenceid=262862255 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/4d94ff98cf9d41a7bff2eb3fa583a11f 2020-11-17 04:34:02,592 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:34:02,610 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/4d94ff98cf9d41a7bff2eb3fa583a11f, entries=417026, sequenceid=262862255, filesize=8.0 M 2020-11-17 04:34:02,611 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~98.18 MB/102947727, heapSize ~128.16 MB/134386736, currentSize=649.88 KB/665473 for e5dc5c620be19b02666a19c280ff582e in 962ms, sequenceid=262862255, compaction requested=true 2020-11-17 04:34:02,613 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 04:34:02,613 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7bc1fb1b75aa4e0da2f2c865abe458c7, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6f48c7563c9b4e0ca57eb88dc130ffb7, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/4d94ff98cf9d41a7bff2eb3fa583a11f] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=76.9 M 2020-11-17 04:34:02,785 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:34:02,789 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:34:02,818 INFO [Thread-1571] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:34:17,583 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#336 average throughput is 61.41 MB/second, slept 17 time(s) and total slept time is 4909 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 04:34:17,665 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 63c8e45b93f842258f55f8cc70fe09fe(size=78.4 M), total size for store is 48.9 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 04:34:17,665 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605587642612; duration=15sec 2020-11-17 04:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.46 MB, max=1.83 GB, blockCount=44172, accesses=14638889, hits=12717617, hitRatio=86.88%, , cachingAccesses=12768847, cachingHits=12714423, cachingHitsRatio=99.57%, evictions=1829, evicted=0, evictedPerRun=0.0 2020-11-17 04:34:39,561 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 55179 ns. 2020-11-17 04:34:58,731 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 46137 ns. 2020-11-17 04:34:58,731 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15217 ns. 2020-11-17 04:35:02,152 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.43 MB heapSize=128.20 MB 2020-11-17 04:35:02,183 INFO [Thread-1576] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:35:02,797 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.43 MB at sequenceid=251779454 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/0b9b68b050ce4329860663933f07db9b 2020-11-17 04:35:02,804 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:35:02,823 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0b9b68b050ce4329860663933f07db9b, entries=311181, sequenceid=251779454, filesize=6.3 M 2020-11-17 04:35:02,825 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.43 MB/111603082, heapSize ~128.20 MB/134424600, currentSize=19.44 KB/19910 for 82421ce8c5c8b48c573baee83caa8305 in 673ms, sequenceid=251779454, compaction requested=true 2020-11-17 04:35:02,826 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 04:35:02,826 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/79329c4839da40d7b740c87a679fe708, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/8b23014d109549208372156b2442de3e, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0b9b68b050ce4329860663933f07db9b] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=57.9 M 2020-11-17 04:35:02,855 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:35:02,857 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:35:02,882 INFO [Thread-1578] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:35:17,493 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587641315 with entries=28446, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587717458 2020-11-17 04:35:19,520 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#338 average throughput is 57.29 MB/second, slept 18 time(s) and total slept time is 7812 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 04:35:19,577 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 35804c33c9e5424ebfbeddb8d0e9c868(size=57.9 M), total size for store is 11.1 G. This selection was in queue for 0sec, and took 16sec to execute. 2020-11-17 04:35:19,577 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605587702825; duration=16sec 2020-11-17 04:35:38,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 349859 ns. 2020-11-17 04:35:38,715 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 102003 ns. 2020-11-17 04:35:38,715 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 80393 ns. 2020-11-17 04:36:39,226 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 52619 ns. 2020-11-17 04:36:47,982 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:37:00,613 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587717458 with entries=38118, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587820590 2020-11-17 04:37:11,167 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.12 MB heapSize=128.00 MB 2020-11-17 04:37:11,203 INFO [Thread-1587] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:37:12,004 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.12 MB at sequenceid=271676240 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/e508c85878a3408686fd4283fa8d50c1 2020-11-17 04:37:12,013 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:37:12,032 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/e508c85878a3408686fd4283fa8d50c1, entries=314510, sequenceid=271676240, filesize=8.7 M 2020-11-17 04:37:12,034 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.12 MB/110227053, heapSize ~128.00 MB/134222104, currentSize=75.23 KB/77036 for 0469ce862509756add4a8cdb00bc465c in 867ms, sequenceid=271676240, compaction requested=true 2020-11-17 04:37:15,633 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.07 MB heapSize=128.42 MB 2020-11-17 04:37:15,656 INFO [Thread-1589] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:37:16,427 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.07 MB at sequenceid=272767936 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/911f87a326234e609b5b886ac36fe3d8 2020-11-17 04:37:16,454 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/911f87a326234e609b5b886ac36fe3d8, entries=279344, sequenceid=272767936, filesize=8.5 M 2020-11-17 04:37:16,456 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.07 MB/113324236, heapSize ~128.42 MB/134660608, currentSize=598.39 KB/612750 for fc93f6dcc564341dcc983bf30971f73d in 823ms, sequenceid=272767936, compaction requested=true 2020-11-17 04:37:16,457 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 04:37:16,457 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/aff66dbaa8094de9a703a54a4c24bbd5, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/509898668d99445ea11b1293ba3e313b, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/911f87a326234e609b5b886ac36fe3d8] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=78.4 M 2020-11-17 04:37:16,509 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:37:16,514 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:37:16,561 INFO [Thread-1591] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:37:17,235 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:37:30,085 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:37:30,176 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:37:32,796 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.51 MB heapSize=128.01 MB 2020-11-17 04:37:32,846 INFO [Thread-1594] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:37:33,527 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#341 average throughput is 57.13 MB/second, slept 18 time(s) and total slept time is 7169 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 04:37:33,639 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.51 MB at sequenceid=262872865 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/0de7bb64a4ef40098f51023443df18d7 2020-11-17 04:37:33,654 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into 7595266df4f4416e85fb345b1a9fa1b2(size=78.8 M), total size for store is 33.3 G. This selection was in queue for 0sec, and took 17sec to execute. 2020-11-17 04:37:33,654 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605587836456; duration=17sec 2020-11-17 04:37:33,663 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/0de7bb64a4ef40098f51023443df18d7, entries=367617, sequenceid=262872865, filesize=8.2 M 2020-11-17 04:37:33,664 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.51 MB/106442171, heapSize ~128.01 MB/134228208, currentSize=691.76 KB/708363 for e5dc5c620be19b02666a19c280ff582e in 868ms, sequenceid=262872865, compaction requested=true 2020-11-17 04:37:38,565 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:37:54,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:38:05,532 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:38:11,285 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:38:16,106 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587820590 with entries=28349, filesize=129.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587896071 2020-11-17 04:38:25,807 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:38:31,450 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:38:38,100 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:38:41,471 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:39:05,544 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:39:13,025 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:39:18,155 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:39:27,641 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:39:34,428 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:39:36,025 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=170.98 MB, max=1.83 GB, blockCount=44235, accesses=15138840, hits=13186365, hitRatio=87.10%, , cachingAccesses=13237671, cachingHits=13183164, cachingHitsRatio=99.59%, evictions=1859, evicted=0, evictedPerRun=0.0 2020-11-17 04:39:46,500 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:40:01,421 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:40:02,537 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587896071 with entries=38540, filesize=128.08 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588002489 2020-11-17 04:40:07,098 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:40:08,721 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 30195 ns. 2020-11-17 04:40:08,722 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 13399 ns. 2020-11-17 04:40:13,187 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:40:19,649 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:40:24,323 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.29 MB heapSize=128.03 MB 2020-11-17 04:40:24,392 INFO [Thread-1603] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:40:25,154 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.29 MB at sequenceid=251789042 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/4c7feac7230b4c05adc37a01300182b2 2020-11-17 04:40:25,166 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:40:25,193 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/4c7feac7230b4c05adc37a01300182b2, entries=310796, sequenceid=251789042, filesize=6.3 M 2020-11-17 04:40:25,195 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.29 MB/111451092, heapSize ~128.03 MB/134248040, currentSize=30.28 KB/31011 for 82421ce8c5c8b48c573baee83caa8305 in 872ms, sequenceid=251789042, compaction requested=true 2020-11-17 04:40:29,476 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:40:37,588 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:40:38,724 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 50601 ns. 2020-11-17 04:40:43,415 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:40:48,874 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.55 MB heapSize=128.05 MB 2020-11-17 04:40:48,943 INFO [Thread-1606] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:40:49,808 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.55 MB at sequenceid=262882600 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/c7779f9868f7432899528b45c5f430f4 2020-11-17 04:40:49,818 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:40:49,858 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c7779f9868f7432899528b45c5f430f4, entries=353483, sequenceid=262882600, filesize=8.0 M 2020-11-17 04:40:49,860 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.55 MB/107530370, heapSize ~128.05 MB/134269288, currentSize=1.25 MB/1313644 for e5dc5c620be19b02666a19c280ff582e in 986ms, sequenceid=262882600, compaction requested=true 2020-11-17 04:40:49,861 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 04:40:49,861 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/13465277ab2247d1a5206bda6b7a3222, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/63c8e45b93f842258f55f8cc70fe09fe, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/0de7bb64a4ef40098f51023443df18d7, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c7779f9868f7432899528b45c5f430f4] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=208.1 M 2020-11-17 04:40:49,964 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:40:49,974 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:40:49,982 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:40:50,089 INFO [Thread-1608] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:41:13,936 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/ced43b19d666462eb54d36eca84943e0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:41:16,453 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588002489 with entries=27582, filesize=129.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588076425 2020-11-17 04:41:27,893 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.00 MB heapSize=128.07 MB 2020-11-17 04:41:27,953 INFO [Thread-1614] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:41:28,679 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.00 MB at sequenceid=259546288 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/950c62292a9f4693b54ab643b9307f86 2020-11-17 04:41:28,684 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:41:28,705 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/950c62292a9f4693b54ab643b9307f86, entries=374975, sequenceid=259546288, filesize=7.1 M 2020-11-17 04:41:28,706 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.00 MB/105910370, heapSize ~128.07 MB/134295472, currentSize=240.91 KB/246694 for 06f7ca9534710e37d90d9f8345864259 in 813ms, sequenceid=259546288, compaction requested=true 2020-11-17 04:41:28,707 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 04:41:28,707 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/144fb4fb493f459990ebff2b637b25d9, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/5acc03e5c6694c73b562b090db235d84, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/950c62292a9f4693b54ab643b9307f86] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=97.2 M 2020-11-17 04:41:28,748 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:41:28,753 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:41:28,782 INFO [Thread-1616] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:41:30,702 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#345 average throughput is 59.53 MB/second, slept 46 time(s) and total slept time is 16088 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 04:41:30,780 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of e5dc5c620be19b02666a19c280ff582e into ced43b19d666462eb54d36eca84943e0(size=211.3 M), total size for store is 48.9 G. This selection was in queue for 0sec, and took 40sec to execute. 2020-11-17 04:41:30,780 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605588049861; duration=40sec 2020-11-17 04:41:38,246 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:41:38,250 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:41:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 111372 ns. 2020-11-17 04:41:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 276592 ns. 2020-11-17 04:41:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 85922 ns. 2020-11-17 04:41:51,149 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#347 average throughput is 57.75 MB/second, slept 23 time(s) and total slept time is 9000 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 04:41:51,240 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into ce5d3fb6914249d19cdb2c3ef2f4d0b3(size=97.4 M), total size for store is 55.4 G. This selection was in queue for 0sec, and took 22sec to execute. 2020-11-17 04:41:51,240 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605588088706; duration=22sec 2020-11-17 04:42:05,457 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:42:12,810 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:42:19,703 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:42:25,047 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.14 MB heapSize=128.00 MB 2020-11-17 04:42:25,102 INFO [Thread-1621] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:42:26,454 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.14 MB at sequenceid=287203977 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/313102ff181447008bd4b8459ee13430 2020-11-17 04:42:26,461 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:42:26,507 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/313102ff181447008bd4b8459ee13430, entries=529010, sequenceid=287203977, filesize=9.1 M 2020-11-17 04:42:26,509 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.14 MB/94521400, heapSize ~128.00 MB/134218472, currentSize=46.35 KB/47463 for 5e5f05205d7e40b0141cd977dd009ba1 in 1462ms, sequenceid=287203977, compaction requested=true 2020-11-17 04:42:26,510 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 04:42:26,511 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/a11e6288393d4dc3bd44c102c195664e, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/d9110d02c75e48699fce82c20bc58e0b, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/313102ff181447008bd4b8459ee13430] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=64.8 M 2020-11-17 04:42:26,592 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:42:26,599 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:42:26,663 INFO [Thread-1623] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:42:36,080 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:42:36,210 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:42:37,714 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:42:38,516 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#349 average throughput is 52.16 MB/second, slept 6 time(s) and total slept time is 1457 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 04:42:38,607 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into c39881a5e79748c69e9687b402db85ed(size=65.4 M), total size for store is 24.3 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 04:42:38,607 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=11, startTime=1605588146509; duration=12sec 2020-11-17 04:42:38,909 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 55566 ns. 2020-11-17 04:43:01,052 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:43:09,796 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588076425 with entries=40983, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588189767 2020-11-17 04:43:09,796 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586601990 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605586601990 2020-11-17 04:43:13,729 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.10 MB heapSize=128.05 MB 2020-11-17 04:43:13,772 INFO [Thread-1626] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:43:14,966 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.10 MB at sequenceid=261393856 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/12e4e1eda2ea4454bf90be27109a9b8c 2020-11-17 04:43:14,973 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:43:15,089 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/12e4e1eda2ea4454bf90be27109a9b8c, entries=580553, sequenceid=261393856, filesize=8.8 M 2020-11-17 04:43:15,092 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.10 MB/89232592, heapSize ~128.05 MB/134266272, currentSize=115.56 KB/118337 for 73aae5715542e1a5370f7540d2117abc in 1363ms, sequenceid=261393856, compaction requested=true 2020-11-17 04:43:35,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:43:39,061 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:44:27,272 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:44:31,691 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588189767 with entries=30468, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588271648 2020-11-17 04:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.15 MB, max=1.83 GB, blockCount=44186, accesses=15740251, hits=13717221, hitRatio=87.15%, , cachingAccesses=13768496, cachingHits=13713949, cachingHitsRatio=99.60%, evictions=1889, evicted=0, evictedPerRun=0.0 2020-11-17 04:44:56,366 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:45:08,988 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 47298 ns. 2020-11-17 04:45:08,988 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 11183 ns. 2020-11-17 04:45:12,197 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:45:32,916 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.60 MB heapSize=128.00 MB 2020-11-17 04:45:32,969 INFO [Thread-1633] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:45:33,915 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.60 MB at sequenceid=262896797 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/80a7fcc2a5e84d2290d0e42ede62b906 2020-11-17 04:45:33,933 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:45:33,956 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/80a7fcc2a5e84d2290d0e42ede62b906, entries=394847, sequenceid=262896797, filesize=8.0 M 2020-11-17 04:45:33,958 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~99.60 MB/104437740, heapSize ~128.00 MB/134221016, currentSize=270.23 KB/276714 for e5dc5c620be19b02666a19c280ff582e in 1043ms, sequenceid=262896797, compaction requested=true 2020-11-17 04:45:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 53051 ns. 2020-11-17 04:45:45,941 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:45:59,838 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:01,358 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.34 MB heapSize=128.23 MB 2020-11-17 04:46:01,384 INFO [Thread-1636] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:01,994 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.34 MB at sequenceid=251798962 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/299c993fe10240edab1b19db96383063 2020-11-17 04:46:02,040 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/299c993fe10240edab1b19db96383063, entries=312881, sequenceid=251798962, filesize=6.4 M 2020-11-17 04:46:02,044 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.34 MB/111509282, heapSize ~128.23 MB/134464056, currentSize=28.38 KB/29060 for 82421ce8c5c8b48c573baee83caa8305 in 686ms, sequenceid=251798962, compaction requested=true 2020-11-17 04:46:02,045 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 04:46:02,046 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/730679d52d49453e845a70db9bcd47b2, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/6b4cf44a87a6405f8670d808835ced56, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/35804c33c9e5424ebfbeddb8d0e9c868, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/4c7feac7230b4c05adc37a01300182b2, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/299c993fe10240edab1b19db96383063] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=422.1 M 2020-11-17 04:46:02,107 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:02,111 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:02,116 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:02,118 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:02,148 INFO [Thread-1638] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:03,992 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:03,994 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:04,061 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:05,076 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:05,079 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:05,156 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:05,233 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:05,312 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:05,394 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:05,469 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:08,366 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588271648 with entries=35687, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588368342 2020-11-17 04:46:08,698 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:08,700 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:09,255 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:09,336 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:09,868 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:09,944 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:10,350 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:21,349 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:32,328 INFO [DataStreamer for file /hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/386e6a28e3be41dd839bfc11f3f92dfb] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:42,855 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:46:46,367 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:05,408 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:13,686 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:13,698 INFO [DataStreamer for file /hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/386e6a28e3be41dd839bfc11f3f92dfb] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:20,919 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588368342 with entries=26932, filesize=128.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588440896 2020-11-17 04:47:25,129 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:25,132 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:25,134 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:25,137 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:25,225 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:25,227 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:25,869 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:28,736 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:28,738 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:28,740 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:28,741 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:28,754 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:38,659 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:38,663 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:38,997 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 142319 ns. 2020-11-17 04:47:38,997 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 252277 ns. 2020-11-17 04:47:38,997 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 71933 ns. 2020-11-17 04:47:38,997 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 32013 ns. 2020-11-17 04:47:39,410 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:39,412 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:40,311 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:40,313 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:40,548 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:40,552 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:40,555 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:42,190 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:42,192 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:43,013 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:43,016 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:43,574 INFO [DataStreamer for file /hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/386e6a28e3be41dd839bfc11f3f92dfb] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:44,514 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:44,516 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:44,588 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:45,825 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:45,828 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:47,063 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#353 average throughput is 57.24 MB/second, slept 103 time(s) and total slept time is 38368 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 04:47:47,111 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:47:47,134 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 5 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 386e6a28e3be41dd839bfc11f3f92dfb(size=413.6 M), total size for store is 11.1 G. This selection was in queue for 0sec, and took 1mins, 45sec to execute. 2020-11-17 04:47:47,134 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605588362045; duration=1mins, 45sec 2020-11-17 04:49:07,224 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588440896 with entries=39331, filesize=128.30 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588547199 2020-11-17 04:49:13,605 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.55 MB heapSize=128.09 MB 2020-11-17 04:49:13,634 INFO [Thread-1652] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:49:14,457 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.55 MB at sequenceid=262908035 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/46884cf704fc41c8b90732b0ca326223 2020-11-17 04:49:14,463 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:49:14,482 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/46884cf704fc41c8b90732b0ca326223, entries=396428, sequenceid=262908035, filesize=8.1 M 2020-11-17 04:49:14,484 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.55 MB/104383341, heapSize ~128.09 MB/134312992, currentSize=712.33 KB/729427 for e5dc5c620be19b02666a19c280ff582e in 879ms, sequenceid=262908035, compaction requested=true 2020-11-17 04:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=170.81 MB, max=1.83 GB, blockCount=44216, accesses=15865004, hits=13744530, hitRatio=86.63%, , cachingAccesses=13795754, cachingHits=13741177, cachingHitsRatio=99.60%, evictions=1919, evicted=0, evictedPerRun=0.0 2020-11-17 04:50:12,590 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.04 MB heapSize=128.00 MB 2020-11-17 04:50:12,632 INFO [Thread-1657] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:13,720 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.04 MB at sequenceid=291106265 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/fcd91d984849446784dab0c444b79f5e 2020-11-17 04:50:13,727 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:13,744 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/fcd91d984849446784dab0c444b79f5e, entries=554821, sequenceid=291106265, filesize=8.7 M 2020-11-17 04:50:13,746 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~88.04 MB/92314457, heapSize ~128.00 MB/134217752, currentSize=80.52 KB/82451 for bdaa31a63b7f3d06d9909b7560333e79 in 1156ms, sequenceid=291106265, compaction requested=true 2020-11-17 04:50:13,747 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-17 04:50:13,747 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/646a21324266408ab1c324f48dd2a24d, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/5ca0cbe824c649b38ed115679eba0157, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/fcd91d984849446784dab0c444b79f5e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp, totalSize=118.5 M 2020-11-17 04:50:13,785 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:13,837 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:13,863 INFO [Thread-1659] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:18,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 17232 ns. 2020-11-17 04:50:18,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 32784 ns. 2020-11-17 04:50:19,658 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588547199 with entries=27578, filesize=128.09 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588619631 2020-11-17 04:50:19,659 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586721199 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605586721199 2020-11-17 04:50:19,666 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586791158 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605586791158 2020-11-17 04:50:19,672 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586910927 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605586910927 2020-11-17 04:50:19,677 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605586991209 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605586991209 2020-11-17 04:50:19,684 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587100225 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605587100225 2020-11-17 04:50:19,690 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587184342 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605587184342 2020-11-17 04:50:19,695 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587285909 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605587285909 2020-11-17 04:50:19,700 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587355155 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605587355155 2020-11-17 04:50:19,706 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587456643 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605587456643 2020-11-17 04:50:19,712 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587536873 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605587536873 2020-11-17 04:50:19,717 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587641315 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605587641315 2020-11-17 04:50:19,722 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587717458 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605587717458 2020-11-17 04:50:23,227 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:25,396 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:28,689 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:28,691 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:28,778 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:30,090 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:33,386 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: bdaa31a63b7f3d06d9909b7560333e79#e#compaction#356 average throughput is 58.65 MB/second, slept 18 time(s) and total slept time is 4276 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 04:50:33,437 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:33,456 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of bdaa31a63b7f3d06d9909b7560333e79 into aad78c6177274ff2b32c71b8be62b898(size=117.4 M), total size for store is 31.8 G. This selection was in queue for 0sec, and took 19sec to execute. 2020-11-17 04:50:33,456 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., storeName=e, priority=11, startTime=1605588613746; duration=19sec 2020-11-17 04:50:38,652 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:39,421 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 56905 ns. 2020-11-17 04:50:42,192 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.34 MB heapSize=128.02 MB 2020-11-17 04:50:42,217 INFO [Thread-1662] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:42,783 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.34 MB at sequenceid=251807297 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/9a5aaa92b00a4b8b9dcc12989811050b 2020-11-17 04:50:42,795 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:42,814 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/9a5aaa92b00a4b8b9dcc12989811050b, entries=309777, sequenceid=251807297, filesize=6.3 M 2020-11-17 04:50:42,816 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.34 MB/111507639, heapSize ~128.02 MB/134233640, currentSize=338.67 KB/346793 for 82421ce8c5c8b48c573baee83caa8305 in 624ms, sequenceid=251807297, compaction requested=true 2020-11-17 04:50:54,186 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.46 MB heapSize=128.57 MB 2020-11-17 04:50:54,229 INFO [Thread-1665] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:55,052 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.46 MB at sequenceid=271716167 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/1340083c577d42138c173db9ec28096d 2020-11-17 04:50:55,059 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:55,077 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/1340083c577d42138c173db9ec28096d, entries=317696, sequenceid=271716167, filesize=8.7 M 2020-11-17 04:50:55,078 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.46 MB/110580982, heapSize ~128.57 MB/134814152, currentSize=28.97 KB/29665 for 0469ce862509756add4a8cdb00bc465c in 892ms, sequenceid=271716167, compaction requested=true 2020-11-17 04:50:55,079 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 04:50:55,079 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/5acc867713164d75b4faa28be29664a8, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/e508c85878a3408686fd4283fa8d50c1, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/1340083c577d42138c173db9ec28096d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=98.7 M 2020-11-17 04:50:55,106 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:55,108 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:50:55,128 INFO [Thread-1667] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:51:11,931 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:51:12,817 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:51:13,714 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#359 average throughput is 61.46 MB/second, slept 21 time(s) and total slept time is 8151 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 04:51:13,812 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 0140f47b490e46efa0c807035be50396(size=99.3 M), total size for store is 42.7 G. This selection was in queue for 0sec, and took 18sec to execute. 2020-11-17 04:51:13,812 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=11, startTime=1605588655078; duration=18sec 2020-11-17 04:51:27,425 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:51:55,485 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.05 MB heapSize=128.00 MB 2020-11-17 04:51:55,518 INFO [Thread-1673] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:51:56,280 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.05 MB at sequenceid=259578986 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/fe13d89bde8f4ce2a975ba301960cb04 2020-11-17 04:51:56,286 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:51:56,304 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/fe13d89bde8f4ce2a975ba301960cb04, entries=371282, sequenceid=259578986, filesize=7.0 M 2020-11-17 04:51:56,306 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.05 MB/105962561, heapSize ~128.00 MB/134218584, currentSize=31.88 KB/32650 for 06f7ca9534710e37d90d9f8345864259 in 821ms, sequenceid=259578986, compaction requested=true 2020-11-17 04:52:03,274 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588619631 with entries=38230, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588723248 2020-11-17 04:52:32,281 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.34 MB heapSize=128.68 MB 2020-11-17 04:52:32,323 INFO [Thread-1676] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:52:33,248 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.34 MB at sequenceid=272795722 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/6e5c85e3a5ba4ba9b56bf0cddb000b3f 2020-11-17 04:52:33,255 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:52:33,274 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/6e5c85e3a5ba4ba9b56bf0cddb000b3f, entries=279169, sequenceid=272795722, filesize=8.5 M 2020-11-17 04:52:33,275 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.34 MB/113603890, heapSize ~128.68 MB/134927736, currentSize=28.27 KB/28948 for fc93f6dcc564341dcc983bf30971f73d in 995ms, sequenceid=272795722, compaction requested=true 2020-11-17 04:52:43,941 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.82 MB heapSize=128.07 MB 2020-11-17 04:52:43,973 INFO [Thread-1679] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:52:44,688 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.82 MB at sequenceid=262918664 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/2ba7287c41514f0392faa02f35048939 2020-11-17 04:52:44,695 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:52:44,712 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2ba7287c41514f0392faa02f35048939, entries=349756, sequenceid=262918664, filesize=8.0 M 2020-11-17 04:52:44,714 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.82 MB/107814287, heapSize ~128.07 MB/134290136, currentSize=1012.19 KB/1036484 for e5dc5c620be19b02666a19c280ff582e in 773ms, sequenceid=262918664, compaction requested=true 2020-11-17 04:52:44,715 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 04:52:44,715 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/80a7fcc2a5e84d2290d0e42ede62b906, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/46884cf704fc41c8b90732b0ca326223, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2ba7287c41514f0392faa02f35048939] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.1 M 2020-11-17 04:52:44,746 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:52:44,748 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:52:44,776 INFO [Thread-1681] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:52:49,665 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#363 average throughput is 61.50 MB/second, slept 5 time(s) and total slept time is 1485 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 04:52:49,769 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into b6ae739bdb654f2eae3a813755649bc7(size=25.1 M), total size for store is 49.0 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 04:52:49,769 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605588764714; duration=5sec 2020-11-17 04:52:54,842 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:53:18,055 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:53:26,725 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588723248 with entries=31066, filesize=128.25 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588806685 2020-11-17 04:53:26,726 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587820590 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605587820590 2020-11-17 04:53:26,740 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605587896071 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605587896071 2020-11-17 04:53:26,748 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588002489 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605588002489 2020-11-17 04:53:38,782 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 39243 ns. 2020-11-17 04:53:38,783 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 204572 ns. 2020-11-17 04:53:38,783 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 139965 ns. 2020-11-17 04:53:38,783 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 125045 ns. 2020-11-17 04:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=178.80 MB, max=1.83 GB, blockCount=44006, accesses=15909938, hits=13747316, hitRatio=86.41%, , cachingAccesses=13798443, cachingHits=13743841, cachingHitsRatio=99.60%, evictions=1949, evicted=0, evictedPerRun=0.0 2020-11-17 04:54:49,542 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.48 MB heapSize=128.01 MB 2020-11-17 04:54:49,586 INFO [Thread-1688] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:54:50,423 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.48 MB at sequenceid=251814660 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/131aacef0ef347e283d3fb8b62b9397b 2020-11-17 04:54:50,433 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:54:50,458 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/131aacef0ef347e283d3fb8b62b9397b, entries=308042, sequenceid=251814660, filesize=6.3 M 2020-11-17 04:54:50,464 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.48 MB/111654449, heapSize ~128.01 MB/134232160, currentSize=26.07 KB/26694 for 82421ce8c5c8b48c573baee83caa8305 in 922ms, sequenceid=251814660, compaction requested=true 2020-11-17 04:54:53,760 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588806685 with entries=31647, filesize=128.16 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588893662 2020-11-17 04:55:18,907 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15597 ns. 2020-11-17 04:55:18,908 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 35289 ns. 2020-11-17 04:55:36,217 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588893662 with entries=15996, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588936191 2020-11-17 04:56:36,364 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:56:38,788 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 56769 ns. 2020-11-17 04:57:11,264 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:57:19,440 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.83 MB heapSize=128.20 MB 2020-11-17 04:57:19,497 INFO [Thread-1696] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:57:20,317 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.83 MB at sequenceid=262932538 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/89e9b3eceb384cdeb6e82bb306953778 2020-11-17 04:57:20,324 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:57:20,345 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/89e9b3eceb384cdeb6e82bb306953778, entries=379918, sequenceid=262932538, filesize=8.1 M 2020-11-17 04:57:20,348 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.83 MB/105723286, heapSize ~128.20 MB/134422336, currentSize=2.00 MB/2092732 for e5dc5c620be19b02666a19c280ff582e in 909ms, sequenceid=262932538, compaction requested=true 2020-11-17 04:57:20,350 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 04:57:20,350 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/fdaad44cfa5c4a12842dcbf2a571c2b0, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d2d14e2dd9bc4ed2bb245097cd27da94, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ced43b19d666462eb54d36eca84943e0, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b6ae739bdb654f2eae3a813755649bc7, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/89e9b3eceb384cdeb6e82bb306953778] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=1.4 G 2020-11-17 04:57:20,430 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:57:20,436 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:57:20,441 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:57:20,453 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:57:20,470 INFO [Thread-1698] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:57:21,894 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:57:28,247 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:57:28,895 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588936191 with entries=42722, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589048867 2020-11-17 04:57:36,114 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:57:39,812 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:57:41,437 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fb7c10a8859e4388997667a45930d8a3] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:57:57,193 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:58:06,460 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fb7c10a8859e4388997667a45930d8a3] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:58:31,552 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fb7c10a8859e4388997667a45930d8a3] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:58:33,777 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:58:54,889 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589048867 with entries=30848, filesize=128.47 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589134841 2020-11-17 04:58:56,127 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fb7c10a8859e4388997667a45930d8a3] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:59:20,481 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fb7c10a8859e4388997667a45930d8a3] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=178.18 MB, max=1.83 GB, blockCount=44018, accesses=16188918, hits=13893334, hitRatio=85.82%, , cachingAccesses=13944416, cachingHits=13889802, cachingHitsRatio=99.61%, evictions=1979, evicted=0, evictedPerRun=0.0 2020-11-17 04:59:38,806 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 313438 ns. 2020-11-17 04:59:38,806 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 126180 ns. 2020-11-17 04:59:38,806 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 164225 ns. 2020-11-17 04:59:38,806 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 44030 ns. 2020-11-17 04:59:44,869 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fb7c10a8859e4388997667a45930d8a3] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:59:56,686 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 04:59:56,688 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:00:05,763 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fb7c10a8859e4388997667a45930d8a3] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:00:06,669 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=86.68 MB heapSize=128.00 MB 2020-11-17 05:00:06,711 INFO [Thread-1714] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:00:08,003 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=86.68 MB at sequenceid=261446271 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/f4c38f14116041acae11aba79f822653 2020-11-17 05:00:08,015 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:00:08,045 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/f4c38f14116041acae11aba79f822653, entries=569719, sequenceid=261446271, filesize=9.0 M 2020-11-17 05:00:08,052 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~86.68 MB/90895408, heapSize ~128.00 MB/134217736, currentSize=41.08 KB/42062 for 73aae5715542e1a5370f7540d2117abc in 1383ms, sequenceid=261446271, compaction requested=true 2020-11-17 05:00:08,055 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 05:00:08,055 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/62b6c407e04c470d8341c4158c4b06d5, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/12e4e1eda2ea4454bf90be27109a9b8c, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/f4c38f14116041acae11aba79f822653] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=81.8 M 2020-11-17 05:00:08,097 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:00:08,102 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:00:08,133 INFO [Thread-1716] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:00:09,894 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:00:10,685 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.64 MB heapSize=128.00 MB 2020-11-17 05:00:10,708 INFO [Thread-1718] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:00:11,562 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.64 MB at sequenceid=251824263 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/0cbfeacbb32a4fec8dfe120621b3865c 2020-11-17 05:00:11,603 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0cbfeacbb32a4fec8dfe120621b3865c, entries=319595, sequenceid=251824263, filesize=6.4 M 2020-11-17 05:00:11,605 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.64 MB/110773844, heapSize ~128.00 MB/134219736, currentSize=232.33 KB/237907 for 82421ce8c5c8b48c573baee83caa8305 in 921ms, sequenceid=251824263, compaction requested=true 2020-11-17 05:00:18,940 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 39628 ns. 2020-11-17 05:00:18,941 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 13415 ns. 2020-11-17 05:00:26,205 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:00:31,273 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#368 average throughput is 31.98 MB/second, slept 14 time(s) and total slept time is 10974 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 05:00:31,356 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:00:31,382 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 2cf6bcb5dbfe41baa7edccf0284973f5(size=82.2 M), total size for store is 12.5 G. This selection was in queue for 0sec, and took 23sec to execute. 2020-11-17 05:00:31,383 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605589208053; duration=23sec 2020-11-17 05:00:31,384 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 05:00:31,384 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/9a5aaa92b00a4b8b9dcc12989811050b, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/131aacef0ef347e283d3fb8b62b9397b, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0cbfeacbb32a4fec8dfe120621b3865c] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=19.0 M 2020-11-17 05:00:31,414 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:00:31,428 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:00:31,456 INFO [Thread-1721] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:00:40,068 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:00:41,414 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#370 average throughput is 31.92 MB/second, slept 6 time(s) and total slept time is 7016 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 05:00:41,499 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 5459fe50a1234ffd9aac553a23ba125b(size=18.9 M), total size for store is 11.1 G. This selection was in queue for 0sec, and took 10sec to execute. 2020-11-17 05:00:41,499 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605589211606; duration=10sec 2020-11-17 05:00:49,169 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589134841 with entries=42145, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589249134 2020-11-17 05:00:50,346 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fb7c10a8859e4388997667a45930d8a3] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:17,795 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fb7c10a8859e4388997667a45930d8a3] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:18,075 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=97.32 MB heapSize=128.11 MB 2020-11-17 05:01:18,108 INFO [Thread-1727] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:19,080 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.32 MB at sequenceid=262944483 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/f89f45044fd2456084a6b8f274acc201 2020-11-17 05:01:19,089 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:19,113 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f89f45044fd2456084a6b8f274acc201, entries=428642, sequenceid=262944483, filesize=8.1 M 2020-11-17 05:01:19,114 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~97.32 MB/102052488, heapSize ~128.11 MB/134331984, currentSize=743.59 KB/761438 for e5dc5c620be19b02666a19c280ff582e in 1039ms, sequenceid=262944483, compaction requested=false 2020-11-17 05:01:24,240 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.96 MB heapSize=128.00 MB 2020-11-17 05:01:24,277 INFO [Thread-1729] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:25,035 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.96 MB at sequenceid=259608925 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/c51de6f2eb554e3994e64293a81c245a 2020-11-17 05:01:25,055 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:25,078 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/c51de6f2eb554e3994e64293a81c245a, entries=373347, sequenceid=259608925, filesize=6.9 M 2020-11-17 05:01:25,079 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.96 MB/105862978, heapSize ~128.00 MB/134219048, currentSize=240.40 KB/246165 for 06f7ca9534710e37d90d9f8345864259 in 839ms, sequenceid=259608925, compaction requested=true 2020-11-17 05:01:25,080 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 05:01:25,080 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/ce5d3fb6914249d19cdb2c3ef2f4d0b3, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/fe13d89bde8f4ce2a975ba301960cb04, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/c51de6f2eb554e3994e64293a81c245a] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=111.4 M 2020-11-17 05:01:25,145 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:25,149 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:25,189 INFO [Thread-1731] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:43,168 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:43,170 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:50,623 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:50,626 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:53,909 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:54,766 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589249134 with entries=24539, filesize=128.34 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589314733 2020-11-17 05:01:54,974 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:55,047 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:56,583 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:56,587 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:58,217 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fb7c10a8859e4388997667a45930d8a3] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:01:59,505 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:02,506 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:04,224 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:04,227 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:04,229 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:04,231 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:04,234 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:04,236 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:04,398 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:04,400 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:05,990 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:11,419 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#373 average throughput is 32.26 MB/second, slept 29 time(s) and total slept time is 29796 ms. 1 active operations remaining, total limit is 65.38 MB/second 2020-11-17 05:02:11,499 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into cf6a51ac359e4d20ac95e35b8cc3bd74(size=111.5 M), total size for store is 55.4 G. This selection was in queue for 0sec, and took 46sec to execute. 2020-11-17 05:02:11,499 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605589285079; duration=46sec 2020-11-17 05:02:14,139 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:17,149 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:22,565 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:22,571 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:22,574 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:22,577 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:22,665 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:22,667 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:24,355 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:24,427 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:26,581 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#366 average throughput is 52.55 MB/second, slept 298 time(s) and total slept time is 131384 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 05:02:26,683 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 5 file(s) in e of e5dc5c620be19b02666a19c280ff582e into fb7c10a8859e4388997667a45930d8a3(size=1.4 G), total size for store is 49.0 G. This selection was in queue for 0sec, and took 5mins, 6sec to execute. 2020-11-17 05:02:26,683 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605589040348; duration=5mins, 6sec 2020-11-17 05:02:32,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:02:39,434 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 50276 ns. 2020-11-17 05:03:18,065 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:03:22,399 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:03:34,417 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589314733 with entries=37223, filesize=128.08 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589414392 2020-11-17 05:03:41,603 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:04:19,035 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.41 MB heapSize=128.02 MB 2020-11-17 05:04:19,087 INFO [Thread-1741] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:04:19,805 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.41 MB at sequenceid=251831731 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/4e765d881b3540cda25d10e1e9cd612e 2020-11-17 05:04:19,811 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:04:19,831 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/4e765d881b3540cda25d10e1e9cd612e, entries=309121, sequenceid=251831731, filesize=6.3 M 2020-11-17 05:04:19,832 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.41 MB/111573783, heapSize ~128.02 MB/134236016, currentSize=612.89 KB/627597 for 82421ce8c5c8b48c573baee83caa8305 in 798ms, sequenceid=251831731, compaction requested=true 2020-11-17 05:04:33,648 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.85 MB heapSize=128.00 MB 2020-11-17 05:04:33,691 INFO [Thread-1744] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:04:34,437 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.85 MB at sequenceid=271755652 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/aba9feb21d2042b09892d8b63837eb8a 2020-11-17 05:04:34,442 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:04:34,458 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/aba9feb21d2042b09892d8b63837eb8a, entries=318294, sequenceid=271755652, filesize=8.7 M 2020-11-17 05:04:34,460 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.85 MB/109942157, heapSize ~128 MB/134217728, currentSize=22.99 KB/23542 for 0469ce862509756add4a8cdb00bc465c in 812ms, sequenceid=271755652, compaction requested=true 2020-11-17 05:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.65 GB, freeSize=186.44 MB, max=1.83 GB, blockCount=43758, accesses=16487279, hits=14021985, hitRatio=85.05%, , cachingAccesses=14072893, cachingHits=14018236, cachingHitsRatio=99.61%, evictions=2009, evicted=0, evictedPerRun=0.0 2020-11-17 05:04:39,172 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 46773 ns. 2020-11-17 05:04:39,817 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.57 MB heapSize=128.29 MB 2020-11-17 05:04:39,848 INFO [Thread-1746] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:04:40,596 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.57 MB at sequenceid=262954781 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/3876ee30911f4b3da97f4b2937963ef7 2020-11-17 05:04:40,602 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:04:40,623 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3876ee30911f4b3da97f4b2937963ef7, entries=356263, sequenceid=262954781, filesize=8.2 M 2020-11-17 05:04:40,625 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.57 MB/107553012, heapSize ~128.29 MB/134518360, currentSize=398.28 KB/407837 for e5dc5c620be19b02666a19c280ff582e in 808ms, sequenceid=262954781, compaction requested=true 2020-11-17 05:04:42,624 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589414392 with entries=25448, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589482607 2020-11-17 05:05:09,161 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:05:19,513 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 36676 ns. 2020-11-17 05:05:19,513 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 13891 ns. 2020-11-17 05:05:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 265804 ns. 2020-11-17 05:05:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 115388 ns. 2020-11-17 05:05:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 137564 ns. 2020-11-17 05:05:43,313 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.06 MB heapSize=128.01 MB 2020-11-17 05:05:43,376 INFO [Thread-1751] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:05:44,412 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.06 MB at sequenceid=287280571 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/73cbd60639f841cab61054c7eb9e8036 2020-11-17 05:05:44,418 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:05:44,437 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/73cbd60639f841cab61054c7eb9e8036, entries=530272, sequenceid=287280571, filesize=9.0 M 2020-11-17 05:05:44,440 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.06 MB/94434452, heapSize ~128.01 MB/134223480, currentSize=33.47 KB/34272 for 5e5f05205d7e40b0141cd977dd009ba1 in 1126ms, sequenceid=287280571, compaction requested=true 2020-11-17 05:06:22,676 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589482607 with entries=37451, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589582619 2020-11-17 05:06:22,677 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588076425 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605588076425 2020-11-17 05:06:22,699 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588189767 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605588189767 2020-11-17 05:06:22,708 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588271648 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605588271648 2020-11-17 05:06:22,713 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588368342 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605588368342 2020-11-17 05:06:22,719 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588440896 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605588440896 2020-11-17 05:07:48,147 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.15 MB heapSize=128.68 MB 2020-11-17 05:07:48,231 INFO [Thread-1759] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:07:48,902 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.15 MB at sequenceid=272823668 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/daef7d3b81394558af198fb50e205ac3 2020-11-17 05:07:48,909 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:07:48,942 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/daef7d3b81394558af198fb50e205ac3, entries=281857, sequenceid=272823668, filesize=8.5 M 2020-11-17 05:07:48,945 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.15 MB/113408660, heapSize ~128.68 MB/134933936, currentSize=342.66 KB/350884 for fc93f6dcc564341dcc983bf30971f73d in 799ms, sequenceid=272823668, compaction requested=true 2020-11-17 05:07:48,947 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 05:07:48,947 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/ea74dd2ad7154f9a841171d85a94d01c, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/e59bacc68ac04eb4875bd2f0bd9cae08, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/7595266df4f4416e85fb345b1a9fa1b2, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/6e5c85e3a5ba4ba9b56bf0cddb000b3f, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/daef7d3b81394558af198fb50e205ac3] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=691.6 M 2020-11-17 05:07:49,019 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:07:49,027 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:07:49,033 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:07:49,037 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:07:49,063 INFO [Thread-1761] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:07:58,004 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589582619 with entries=33990, filesize=128.08 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589677975 2020-11-17 05:07:58,043 WARN [Close-WAL-Writer-195] asyncfs.FanOutOneBlockAsyncDFSOutputHelper: complete file /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589582619 not finished, retry = 0 2020-11-17 05:08:17,927 INFO [DataStreamer for file /hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/1b97c15eb4f445c49c9014fc038717d4] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:08:38,760 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 51802 ns. 2020-11-17 05:08:47,297 INFO [DataStreamer for file /hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/1b97c15eb4f445c49c9014fc038717d4] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:16,557 INFO [DataStreamer for file /hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/1b97c15eb4f445c49c9014fc038717d4] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:18,052 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.63 MB heapSize=128.00 MB 2020-11-17 05:09:18,101 INFO [Thread-1771] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:18,877 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.63 MB at sequenceid=262968578 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/2e23cd72096946c98e9810b6ecac7e10 2020-11-17 05:09:18,884 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:18,903 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2e23cd72096946c98e9810b6ecac7e10, entries=352053, sequenceid=262968578, filesize=8.1 M 2020-11-17 05:09:18,906 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.63 MB/107614422, heapSize ~128.00 MB/134218488, currentSize=26.83 KB/27469 for e5dc5c620be19b02666a19c280ff582e in 854ms, sequenceid=262968578, compaction requested=true 2020-11-17 05:09:18,907 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 05:09:18,908 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f89f45044fd2456084a6b8f274acc201, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3876ee30911f4b3da97f4b2937963ef7, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2e23cd72096946c98e9810b6ecac7e10] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.3 M 2020-11-17 05:09:18,944 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:18,946 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:18,975 INFO [Thread-1773] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:28,748 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#381 average throughput is 30.89 MB/second, slept 6 time(s) and total slept time is 4895 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 05:09:28,808 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 1db1a5d081574e088870f78529c4597c(size=25.3 M), total size for store is 49.0 G. This selection was in queue for 0sec, and took 9sec to execute. 2020-11-17 05:09:28,808 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605589758906; duration=9sec 2020-11-17 05:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.65 GB, freeSize=186.30 MB, max=1.83 GB, blockCount=43762, accesses=16591452, hits=14025370, hitRatio=84.53%, , cachingAccesses=14076282, cachingHits=14021621, cachingHitsRatio=99.61%, evictions=2039, evicted=0, evictedPerRun=0.0 2020-11-17 05:09:41,736 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:41,752 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:41,754 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:41,756 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:42,237 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:42,240 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:42,505 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:42,507 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:42,734 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:42,737 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:43,888 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:43,890 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:44,446 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:44,448 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:44,890 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:45,010 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:45,206 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:45,208 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:46,343 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:46,442 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:46,523 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:46,601 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:46,681 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:46,762 INFO [DataStreamer for file /hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/1b97c15eb4f445c49c9014fc038717d4] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:46,765 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:48,678 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:49,322 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:49,324 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:49,326 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:49,328 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:50,443 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:50,540 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:50,542 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:50,544 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:50,546 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:50,795 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:51,569 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:55,802 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:55,804 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:56,585 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:56,587 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:09:56,815 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:10:00,014 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:10:00,017 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:10:00,019 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:10:00,021 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:10:00,024 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:10:00,025 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:10:04,044 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:10:08,216 INFO [DataStreamer for file /hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/1b97c15eb4f445c49c9014fc038717d4] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:10:11,874 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:10:12,636 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:10:12,870 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:10:13,059 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:10:13,114 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#379 average throughput is 56.81 MB/second, slept 154 time(s) and total slept time is 70878 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 05:10:13,191 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 5 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into 1b97c15eb4f445c49c9014fc038717d4(size=669.0 M), total size for store is 33.3 G. This selection was in queue for 0sec, and took 2mins, 24sec to execute. 2020-11-17 05:10:13,192 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605589668946; duration=2mins, 24sec 2020-11-17 05:10:29,216 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14710 ns. 2020-11-17 05:10:29,216 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 39347 ns. 2020-11-17 05:10:35,060 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589677975 with entries=57411, filesize=128.26 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589835031 2020-11-17 05:10:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 49334 ns. 2020-11-17 05:10:44,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:11:10,410 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:11:39,210 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 76856 ns. 2020-11-17 05:11:39,210 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 234392 ns. 2020-11-17 05:11:39,210 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 74548 ns. 2020-11-17 05:12:07,031 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:12:13,839 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:12:22,312 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:12:28,662 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:12:35,217 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:12:53,212 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:12:56,370 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589835031 with entries=50139, filesize=128.56 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589976335 2020-11-17 05:12:57,792 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:13:05,568 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:13:15,421 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:13:17,923 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=97.71 MB heapSize=128.05 MB 2020-11-17 05:13:17,969 INFO [Thread-1786] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:13:18,801 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:13:19,043 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.71 MB at sequenceid=262980277 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/bebcaae08e8f4b78a00f6cc3f98749d4 2020-11-17 05:13:19,079 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/bebcaae08e8f4b78a00f6cc3f98749d4, entries=422947, sequenceid=262980277, filesize=8.0 M 2020-11-17 05:13:19,081 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~97.71 MB/102454629, heapSize ~128.05 MB/134273192, currentSize=689.14 KB/705684 for e5dc5c620be19b02666a19c280ff582e in 1159ms, sequenceid=262980277, compaction requested=true 2020-11-17 05:13:27,353 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:13:38,770 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 37937 ns. 2020-11-17 05:13:48,931 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:13:52,308 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:13:55,498 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:14:03,879 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:14:12,103 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:14:16,613 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=174.21 MB, max=1.83 GB, blockCount=44166, accesses=17388947, hits=14786076, hitRatio=85.03%, , cachingAccesses=14837443, cachingHits=14782235, cachingHitsRatio=99.63%, evictions=2069, evicted=0, evictedPerRun=0.0 2020-11-17 05:14:39,161 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589976335 with entries=36863, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590079117 2020-11-17 05:15:10,924 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:15:16,769 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:15:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 10904 ns. 2020-11-17 05:15:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 43898 ns. 2020-11-17 05:16:08,765 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.05 MB heapSize=128.01 MB 2020-11-17 05:16:08,808 INFO [Thread-1794] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:16:09,646 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:16:10,170 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.05 MB at sequenceid=261495489 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/b2b7322d93c441f48e4b22c49613294b 2020-11-17 05:16:10,195 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/b2b7322d93c441f48e4b22c49613294b, entries=584849, sequenceid=261495489, filesize=9.0 M 2020-11-17 05:16:10,196 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.05 MB/89178598, heapSize ~128.01 MB/134233440, currentSize=117.02 KB/119824 for 73aae5715542e1a5370f7540d2117abc in 1431ms, sequenceid=261495489, compaction requested=true 2020-11-17 05:16:32,813 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:16:37,605 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.35 MB heapSize=128.07 MB 2020-11-17 05:16:37,662 INFO [Thread-1797] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:16:38,400 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.35 MB at sequenceid=262990139 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/5be531bf579345bda9825441605297e0 2020-11-17 05:16:38,406 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:16:38,427 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5be531bf579345bda9825441605297e0, entries=356928, sequenceid=262990139, filesize=8.2 M 2020-11-17 05:16:38,429 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.35 MB/107321152, heapSize ~128.07 MB/134286552, currentSize=487.84 KB/499544 for e5dc5c620be19b02666a19c280ff582e in 824ms, sequenceid=262990139, compaction requested=true 2020-11-17 05:16:38,430 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 05:16:38,430 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1db1a5d081574e088870f78529c4597c, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/bebcaae08e8f4b78a00f6cc3f98749d4, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5be531bf579345bda9825441605297e0] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.5 M 2020-11-17 05:16:38,461 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:16:38,463 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:16:38,497 INFO [Thread-1799] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:16:38,750 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 38846 ns. 2020-11-17 05:16:47,198 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#385 average throughput is 57.62 MB/second, slept 10 time(s) and total slept time is 2979 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 05:16:47,286 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 886cf406b4354a81ba0dccd00c1eef11(size=42.7 M), total size for store is 49.0 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 05:16:47,286 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605590198429; duration=8sec 2020-11-17 05:16:51,052 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590079117 with entries=47628, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590210927 2020-11-17 05:17:21,548 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:17:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 208027 ns. 2020-11-17 05:17:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 49612 ns. 2020-11-17 05:17:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 92605 ns. 2020-11-17 05:17:55,945 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:18:01,041 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:18:04,369 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.00 MB heapSize=128.00 MB 2020-11-17 05:18:04,407 INFO [Thread-1805] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:18:05,146 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.00 MB at sequenceid=271794630 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/0ae1eeee628f47f7bd86d594ab12729c 2020-11-17 05:18:05,154 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:18:05,183 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/0ae1eeee628f47f7bd86d594ab12729c, entries=316128, sequenceid=271794630, filesize=8.7 M 2020-11-17 05:18:05,184 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.00 MB/110098001, heapSize ~128.00 MB/134218264, currentSize=29.25 KB/29950 for 0469ce862509756add4a8cdb00bc465c in 815ms, sequenceid=271794630, compaction requested=true 2020-11-17 05:18:05,185 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 05:18:05,185 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/0140f47b490e46efa0c807035be50396, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/aba9feb21d2042b09892d8b63837eb8a, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/0ae1eeee628f47f7bd86d594ab12729c] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=116.6 M 2020-11-17 05:18:05,219 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:18:05,226 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:18:05,248 INFO [Thread-1807] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:18:13,389 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:18:26,192 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:18:29,972 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#387 average throughput is 54.51 MB/second, slept 25 time(s) and total slept time is 12516 ms. 0 active operations remaining, total limit is 53.85 MB/second 2020-11-17 05:18:30,049 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into dbe3768d869b4062a2cc5af792658c37(size=117.1 M), total size for store is 42.7 G. This selection was in queue for 0sec, and took 24sec to execute. 2020-11-17 05:18:30,049 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=11, startTime=1605590285185; duration=24sec 2020-11-17 05:18:31,665 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=99.02 MB heapSize=128.00 MB 2020-11-17 05:18:31,698 INFO [Thread-1810] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:18:32,463 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.02 MB at sequenceid=259661722 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/4bf5de5997a74ba498ee9ffb8bbfe09d 2020-11-17 05:18:32,485 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/4bf5de5997a74ba498ee9ffb8bbfe09d, entries=402208, sequenceid=259661722, filesize=7.8 M 2020-11-17 05:18:32,486 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~99.02 MB/103831717, heapSize ~128.00 MB/134220200, currentSize=63.36 KB/64881 for 06f7ca9534710e37d90d9f8345864259 in 821ms, sequenceid=259661722, compaction requested=true 2020-11-17 05:18:42,555 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:18:49,111 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:18:57,041 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:19:09,772 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:19:17,315 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590210927 with entries=54067, filesize=128.14 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590357285 2020-11-17 05:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=174.02 MB, max=1.83 GB, blockCount=44172, accesses=17538743, hits=14905899, hitRatio=84.99%, , cachingAccesses=14957245, cachingHits=14902014, cachingHitsRatio=99.63%, evictions=2099, evicted=0, evictedPerRun=0.0 2020-11-17 05:19:38,719 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 43031 ns. 2020-11-17 05:20:25,322 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.83 MB heapSize=128.48 MB 2020-11-17 05:20:25,367 INFO [Thread-1817] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:20:26,057 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.83 MB at sequenceid=263001413 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/135a7b9768104b51928c15ade8b91d8d 2020-11-17 05:20:26,062 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:20:26,077 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/135a7b9768104b51928c15ade8b91d8d, entries=356161, sequenceid=263001413, filesize=8.3 M 2020-11-17 05:20:26,078 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.83 MB/107825053, heapSize ~128.48 MB/134724528, currentSize=740.43 KB/758196 for e5dc5c620be19b02666a19c280ff582e in 756ms, sequenceid=263001413, compaction requested=true 2020-11-17 05:20:38,764 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14491 ns. 2020-11-17 05:20:38,765 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 30496 ns. 2020-11-17 05:20:42,165 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:21:15,829 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590357285 with entries=41798, filesize=128.67 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590475769 2020-11-17 05:22:38,657 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:22:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 36010 ns. 2020-11-17 05:22:53,478 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.43 MB heapSize=128.00 MB 2020-11-17 05:22:53,526 INFO [Thread-1826] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:22:54,666 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.43 MB at sequenceid=291207855 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/992eaf481d8243a49134bf2faa07c945 2020-11-17 05:22:54,675 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:22:54,697 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/992eaf481d8243a49134bf2faa07c945, entries=549921, sequenceid=291207855, filesize=9.0 M 2020-11-17 05:22:54,699 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~88.43 MB/92728866, heapSize ~128.00 MB/134219608, currentSize=40.39 KB/41359 for bdaa31a63b7f3d06d9909b7560333e79 in 1221ms, sequenceid=291207855, compaction requested=true 2020-11-17 05:23:12,770 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:23:18,065 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:23:24,698 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590475769 with entries=47126, filesize=128.17 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590604659 2020-11-17 05:23:24,699 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588547199 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605588547199 2020-11-17 05:23:24,707 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588619631 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605588619631 2020-11-17 05:23:24,712 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588723248 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605588723248 2020-11-17 05:23:24,717 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588806685 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605588806685 2020-11-17 05:23:24,723 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588893662 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605588893662 2020-11-17 05:23:24,728 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605588936191 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605588936191 2020-11-17 05:23:24,734 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589048867 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605589048867 2020-11-17 05:23:24,738 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589134841 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605589134841 2020-11-17 05:23:24,743 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589249134 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605589249134 2020-11-17 05:23:24,748 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589314733 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605589314733 2020-11-17 05:23:35,173 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.08 MB heapSize=128.03 MB 2020-11-17 05:23:35,195 INFO [Thread-1829] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:23:35,876 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.08 MB at sequenceid=272851752 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/879c31fb7e1c48268c7e1d71dd97c1e5 2020-11-17 05:23:35,882 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:23:35,899 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/879c31fb7e1c48268c7e1d71dd97c1e5, entries=273485, sequenceid=272851752, filesize=8.5 M 2020-11-17 05:23:35,900 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.08 MB/113328649, heapSize ~128.03 MB/134249312, currentSize=668.80 KB/684850 for fc93f6dcc564341dcc983bf30971f73d in 727ms, sequenceid=272851752, compaction requested=true 2020-11-17 05:23:38,755 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 141942 ns. 2020-11-17 05:23:38,755 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 184453 ns. 2020-11-17 05:23:38,755 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 43405 ns. 2020-11-17 05:23:58,969 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:24:25,896 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:24:33,391 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=174.86 MB, max=1.83 GB, blockCount=44148, accesses=17546098, hits=14913235, hitRatio=84.99%, , cachingAccesses=14964600, cachingHits=14909350, cachingHitsRatio=99.63%, evictions=2129, evicted=0, evictedPerRun=0.0 2020-11-17 05:25:22,003 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.38 MB heapSize=128.09 MB 2020-11-17 05:25:22,049 INFO [Thread-1835] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:25:22,787 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.38 MB at sequenceid=263016246 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/8e69dcbe4ac04fef808956cea94c9cc3 2020-11-17 05:25:22,794 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:25:22,811 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8e69dcbe4ac04fef808956cea94c9cc3, entries=399841, sequenceid=263016246, filesize=8.1 M 2020-11-17 05:25:22,812 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.38 MB/104208646, heapSize ~128.09 MB/134310744, currentSize=488.26 KB/499983 for e5dc5c620be19b02666a19c280ff582e in 809ms, sequenceid=263016246, compaction requested=true 2020-11-17 05:25:22,813 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 05:25:22,814 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/886cf406b4354a81ba0dccd00c1eef11, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/135a7b9768104b51928c15ade8b91d8d, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8e69dcbe4ac04fef808956cea94c9cc3] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=59.2 M 2020-11-17 05:25:22,841 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:25:22,843 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:25:22,871 INFO [Thread-1837] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:25:35,052 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#393 average throughput is 57.77 MB/second, slept 14 time(s) and total slept time is 4158 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 05:25:35,133 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 9d69cab2c33b4478b959b9b19e1c33be(size=60.7 M), total size for store is 49.0 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 05:25:35,133 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605590722812; duration=12sec 2020-11-17 05:25:39,226 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 27583 ns. 2020-11-17 05:25:39,226 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 54090 ns. 2020-11-17 05:25:39,227 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 35226 ns. 2020-11-17 05:25:51,993 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590604659 with entries=54070, filesize=128.10 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590751942 2020-11-17 05:26:20,611 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:27:35,092 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=103.04 MB heapSize=128.00 MB 2020-11-17 05:27:35,125 INFO [Thread-1846] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:27:35,933 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.04 MB at sequenceid=251871176 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/53b2f27b3e864c12952a1028217fb0c5 2020-11-17 05:27:35,942 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:27:35,961 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/53b2f27b3e864c12952a1028217fb0c5, entries=354000, sequenceid=251871176, filesize=7.3 M 2020-11-17 05:27:35,963 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.04 MB/108041109, heapSize ~128.00 MB/134218192, currentSize=22.24 KB/22774 for 82421ce8c5c8b48c573baee83caa8305 in 871ms, sequenceid=251871176, compaction requested=true 2020-11-17 05:27:35,964 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 05:27:35,965 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/5459fe50a1234ffd9aac553a23ba125b, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/4e765d881b3540cda25d10e1e9cd612e, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/53b2f27b3e864c12952a1028217fb0c5] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=32.5 M 2020-11-17 05:27:36,012 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:27:36,016 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:27:36,044 INFO [Thread-1848] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:27:39,531 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 41203 ns. 2020-11-17 05:27:43,837 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590751942 with entries=41589, filesize=128.14 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590863808 2020-11-17 05:27:43,838 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589414392 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605589414392 2020-11-17 05:27:45,090 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#395 average throughput is 58.21 MB/second, slept 10 time(s) and total slept time is 3769 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 05:27:45,147 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into fef74e470f644845a476137770185838(size=32.5 M), total size for store is 11.1 G. This selection was in queue for 0sec, and took 9sec to execute. 2020-11-17 05:27:45,147 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605590855963; duration=9sec 2020-11-17 05:28:03,005 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:28:07,122 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:28:21,860 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:28:29,762 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:28:46,628 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.22 MB heapSize=128.06 MB 2020-11-17 05:28:46,678 INFO [Thread-1853] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:28:47,595 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.22 MB at sequenceid=263026462 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/4b77c61be3ec4374bb443d2ea90e91f9 2020-11-17 05:28:47,604 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:28:47,625 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/4b77c61be3ec4374bb443d2ea90e91f9, entries=372683, sequenceid=263026462, filesize=8.2 M 2020-11-17 05:28:47,631 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.22 MB/106133793, heapSize ~128.06 MB/134279392, currentSize=712.01 KB/729098 for e5dc5c620be19b02666a19c280ff582e in 1003ms, sequenceid=263026462, compaction requested=true 2020-11-17 05:29:08,311 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590863808 with entries=30632, filesize=128.14 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590948263 2020-11-17 05:29:22,908 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=174.44 MB, max=1.83 GB, blockCount=44159, accesses=17699856, hits=15046974, hitRatio=85.01%, , cachingAccesses=15098349, cachingHits=15043082, cachingHitsRatio=99.63%, evictions=2159, evicted=0, evictedPerRun=0.0 2020-11-17 05:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 05:29:39,505 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 53985 ns. 2020-11-17 05:29:39,506 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 139615 ns. 2020-11-17 05:29:39,506 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 203694 ns. 2020-11-17 05:30:31,558 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:30:48,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 16703 ns. 2020-11-17 05:30:48,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 22662 ns. 2020-11-17 05:30:58,426 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.71 MB heapSize=128.00 MB 2020-11-17 05:30:58,480 INFO [Thread-1860] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:30:59,655 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.71 MB at sequenceid=287362044 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/969e7820250b45aa972a3e4d5f3b7887 2020-11-17 05:30:59,663 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:30:59,681 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/969e7820250b45aa972a3e4d5f3b7887, entries=520464, sequenceid=287362044, filesize=9.2 M 2020-11-17 05:30:59,684 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.71 MB/95112614, heapSize ~128.00 MB/134218048, currentSize=520.30 KB/532791 for 5e5f05205d7e40b0141cd977dd009ba1 in 1259ms, sequenceid=287362044, compaction requested=true 2020-11-17 05:30:59,685 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 05:30:59,685 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/c39881a5e79748c69e9687b402db85ed, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/73cbd60639f841cab61054c7eb9e8036, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/969e7820250b45aa972a3e4d5f3b7887] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=83.7 M 2020-11-17 05:30:59,748 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:30:59,752 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:30:59,780 INFO [Thread-1862] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:30:59,942 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590948263 with entries=41705, filesize=128.17 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591059891 2020-11-17 05:30:59,943 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589482607 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605589482607 2020-11-17 05:30:59,963 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589582619 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605589582619 2020-11-17 05:30:59,971 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589677975 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605589677975 2020-11-17 05:30:59,986 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589835031 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605589835031 2020-11-17 05:30:59,990 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605589976335 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605589976335 2020-11-17 05:31:04,536 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:31:04,544 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:31:10,881 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:31:11,075 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:31:11,077 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:31:12,783 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:31:12,785 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:31:13,792 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#398 average throughput is 56.40 MB/second, slept 13 time(s) and total slept time is 2701 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 05:31:13,890 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into d2e0233766f74cc084f39b4eadd1eb59(size=84.2 M), total size for store is 24.3 G. This selection was in queue for 0sec, and took 14sec to execute. 2020-11-17 05:31:13,890 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=11, startTime=1605591059684; duration=14sec 2020-11-17 05:31:38,723 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 58850 ns. 2020-11-17 05:31:48,652 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:31:49,533 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.63 MB heapSize=128.86 MB 2020-11-17 05:31:49,569 INFO [Thread-1867] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:31:50,398 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.63 MB at sequenceid=271834439 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/ad5a26343b0c4aa19b0c0d74d5179935 2020-11-17 05:31:50,421 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/ad5a26343b0c4aa19b0c0d74d5179935, entries=319266, sequenceid=271834439, filesize=8.7 M 2020-11-17 05:31:50,422 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.63 MB/110765148, heapSize ~128.86 MB/135117944, currentSize=27.50 KB/28165 for 0469ce862509756add4a8cdb00bc465c in 889ms, sequenceid=271834439, compaction requested=true 2020-11-17 05:31:52,433 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.43 MB heapSize=128.07 MB 2020-11-17 05:31:52,463 INFO [Thread-1869] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:31:53,619 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.43 MB at sequenceid=261544255 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/923e4baf851d44699c4a58dd3e13177a 2020-11-17 05:31:53,626 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:31:53,650 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/923e4baf851d44699c4a58dd3e13177a, entries=577397, sequenceid=261544255, filesize=9.0 M 2020-11-17 05:31:53,652 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.43 MB/89583137, heapSize ~128.07 MB/134293872, currentSize=134.30 KB/137527 for 73aae5715542e1a5370f7540d2117abc in 1219ms, sequenceid=261544255, compaction requested=true 2020-11-17 05:31:53,653 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 05:31:53,653 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/2cf6bcb5dbfe41baa7edccf0284973f5, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/b2b7322d93c441f48e4b22c49613294b, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/923e4baf851d44699c4a58dd3e13177a] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=100.1 M 2020-11-17 05:31:53,693 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:31:53,699 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:31:53,721 INFO [Thread-1871] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:32:03,256 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:32:06,236 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.80 MB heapSize=128.03 MB 2020-11-17 05:32:06,272 INFO [Thread-1874] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:32:07,020 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591059891 with entries=24540, filesize=128.47 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591126988 2020-11-17 05:32:07,021 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590079117 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605590079117 2020-11-17 05:32:07,209 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.80 MB at sequenceid=263036561 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/82bc89f29a6b481bb04b4ac8b0496d08 2020-11-17 05:32:07,217 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:32:07,245 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/82bc89f29a6b481bb04b4ac8b0496d08, entries=349872, sequenceid=263036561, filesize=8.1 M 2020-11-17 05:32:07,251 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.80 MB/107792677, heapSize ~128.03 MB/134254240, currentSize=1.06 MB/1115769 for e5dc5c620be19b02666a19c280ff582e in 1015ms, sequenceid=263036561, compaction requested=true 2020-11-17 05:32:07,252 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 05:32:07,252 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/9d69cab2c33b4478b959b9b19e1c33be, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/4b77c61be3ec4374bb443d2ea90e91f9, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/82bc89f29a6b481bb04b4ac8b0496d08] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=77.1 M 2020-11-17 05:32:07,293 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:32:07,296 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:32:07,323 INFO [Thread-1876] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:32:12,134 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#401 average throughput is 48.98 MB/second, slept 14 time(s) and total slept time is 5074 ms. 1 active operations remaining, total limit is 57.69 MB/second 2020-11-17 05:32:12,194 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:32:12,228 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 2af08ab4a1d94819bcc817ad9dbd4a2d(size=100.4 M), total size for store is 12.5 G. This selection was in queue for 0sec, and took 18sec to execute. 2020-11-17 05:32:12,228 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605591113652; duration=18sec 2020-11-17 05:32:18,445 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:32:26,200 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#403 average throughput is 48.10 MB/second, slept 16 time(s) and total slept time is 8399 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 05:32:26,281 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into d365f50d37f24e1198282234d62dd603(size=78.5 M), total size for store is 49.0 G. This selection was in queue for 0sec, and took 19sec to execute. 2020-11-17 05:32:26,281 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605591127251; duration=19sec 2020-11-17 05:33:15,317 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.61 MB heapSize=128.07 MB 2020-11-17 05:33:15,364 INFO [Thread-1881] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:33:15,990 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.61 MB at sequenceid=251881073 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/58f738ad614944899611a039f3045605 2020-11-17 05:33:15,998 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:33:16,017 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/58f738ad614944899611a039f3045605, entries=307161, sequenceid=251881073, filesize=6.4 M 2020-11-17 05:33:16,019 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.61 MB/111787649, heapSize ~128.07 MB/134293928, currentSize=17.32 KB/17740 for 82421ce8c5c8b48c573baee83caa8305 in 702ms, sequenceid=251881073, compaction requested=true 2020-11-17 05:33:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 38426 ns. 2020-11-17 05:33:58,266 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591126988 with entries=40929, filesize=128.51 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591238233 2020-11-17 05:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=176.52 MB, max=1.83 GB, blockCount=44099, accesses=17863909, hits=15168401, hitRatio=84.91%, , cachingAccesses=15219734, cachingHits=15164445, cachingHitsRatio=99.64%, evictions=2189, evicted=0, evictedPerRun=0.0 2020-11-17 05:35:11,144 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591238233 with entries=26721, filesize=128.21 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591311101 2020-11-17 05:35:11,398 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=98.75 MB heapSize=128.04 MB 2020-11-17 05:35:11,440 INFO [Thread-1887] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:35:12,373 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.75 MB at sequenceid=259713467 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/f6f7ba50088a457c918b846dd9289d21 2020-11-17 05:35:12,380 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:35:12,405 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/f6f7ba50088a457c918b846dd9289d21, entries=396528, sequenceid=259713467, filesize=7.6 M 2020-11-17 05:35:12,407 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.75 MB/103548339, heapSize ~128.04 MB/134257400, currentSize=214.80 KB/219955 for 06f7ca9534710e37d90d9f8345864259 in 1008ms, sequenceid=259713467, compaction requested=true 2020-11-17 05:35:12,407 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 05:35:12,408 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/cf6a51ac359e4d20ac95e35b8cc3bd74, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/4bf5de5997a74ba498ee9ffb8bbfe09d, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/f6f7ba50088a457c918b846dd9289d21] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=126.9 M 2020-11-17 05:35:12,443 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:35:12,448 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:35:12,482 INFO [Thread-1889] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:35:22,451 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:35:24,444 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:35:24,447 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:35:31,704 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:35:38,736 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 182262 ns. 2020-11-17 05:35:38,737 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 57851 ns. 2020-11-17 05:35:38,737 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 87852 ns. 2020-11-17 05:35:42,008 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#406 average throughput is 56.95 MB/second, slept 30 time(s) and total slept time is 10932 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 05:35:42,093 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 28b36149ece54e1ebe807c2e6ea08e56(size=126.9 M), total size for store is 55.4 G. This selection was in queue for 0sec, and took 29sec to execute. 2020-11-17 05:35:42,093 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605591312407; duration=29sec 2020-11-17 05:35:47,387 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:35:59,180 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 17811 ns. 2020-11-17 05:35:59,180 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 35083 ns. 2020-11-17 05:36:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 54248 ns. 2020-11-17 05:36:48,000 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:36:50,044 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.03 MB heapSize=128.05 MB 2020-11-17 05:36:50,079 INFO [Thread-1896] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:36:51,000 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.03 MB at sequenceid=263050834 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/ebd97d1b8e6249bcb0f11b2ec59ed08a 2020-11-17 05:36:51,023 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ebd97d1b8e6249bcb0f11b2ec59ed08a, entries=375263, sequenceid=263050834, filesize=8.1 M 2020-11-17 05:36:51,025 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.03 MB/105933870, heapSize ~128.05 MB/134266512, currentSize=233.36 KB/238962 for e5dc5c620be19b02666a19c280ff582e in 981ms, sequenceid=263050834, compaction requested=true 2020-11-17 05:37:04,821 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591311101 with entries=42382, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591424790 2020-11-17 05:37:04,822 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590210927 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605590210927 2020-11-17 05:37:04,833 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590357285 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605590357285 2020-11-17 05:37:40,548 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:38:14,995 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591424790 with entries=26848, filesize=128.39 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591494956 2020-11-17 05:38:39,249 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 48369 ns. 2020-11-17 05:38:46,963 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.50 MB heapSize=128.19 MB 2020-11-17 05:38:47,005 INFO [Thread-1904] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:38:47,629 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.50 MB at sequenceid=251890944 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/da5428dc8171409d8af6cd7ac2a937a1 2020-11-17 05:38:47,636 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:38:47,658 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/da5428dc8171409d8af6cd7ac2a937a1, entries=310061, sequenceid=251890944, filesize=6.4 M 2020-11-17 05:38:47,660 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.50 MB/111669004, heapSize ~128.19 MB/134414416, currentSize=17.10 KB/17508 for 82421ce8c5c8b48c573baee83caa8305 in 697ms, sequenceid=251890944, compaction requested=true 2020-11-17 05:38:47,661 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 05:38:47,661 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/fef74e470f644845a476137770185838, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/58f738ad614944899611a039f3045605, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/da5428dc8171409d8af6cd7ac2a937a1] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=45.2 M 2020-11-17 05:38:47,702 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:38:47,705 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:38:47,734 INFO [Thread-1906] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:39:01,431 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#409 average throughput is 54.00 MB/second, slept 14 time(s) and total slept time is 6902 ms. 0 active operations remaining, total limit is 53.85 MB/second 2020-11-17 05:39:01,512 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 0a918c8b3c864fdf98102e1782bc4898(size=45.2 M), total size for store is 11.1 G. This selection was in queue for 0sec, and took 13sec to execute. 2020-11-17 05:39:01,512 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605591527660; duration=13sec 2020-11-17 05:39:10,315 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.90 MB heapSize=128.00 MB 2020-11-17 05:39:10,341 INFO [Thread-1909] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:39:11,095 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.90 MB at sequenceid=272880397 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/058b704583dc4313b721c72e43e011dc 2020-11-17 05:39:11,101 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:39:11,119 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/058b704583dc4313b721c72e43e011dc, entries=275861, sequenceid=272880397, filesize=8.5 M 2020-11-17 05:39:11,121 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~107.90 MB/113146063, heapSize ~128.00 MB/134221128, currentSize=26.43 KB/27062 for fc93f6dcc564341dcc983bf30971f73d in 806ms, sequenceid=272880397, compaction requested=true 2020-11-17 05:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=176.85 MB, max=1.83 GB, blockCount=44092, accesses=17906184, hits=15171283, hitRatio=84.73%, , cachingAccesses=15222624, cachingHits=15167316, cachingHitsRatio=99.64%, evictions=2219, evicted=0, evictedPerRun=0.0 2020-11-17 05:40:01,336 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591494956 with entries=38471, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591601303 2020-11-17 05:40:26,247 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.21 MB heapSize=128.00 MB 2020-11-17 05:40:26,294 INFO [Thread-1914] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:40:27,065 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.21 MB at sequenceid=263061866 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/15b816f3416843cb8e67417918713bb0 2020-11-17 05:40:27,072 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:40:27,090 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/15b816f3416843cb8e67417918713bb0, entries=414464, sequenceid=263061866, filesize=8.1 M 2020-11-17 05:40:27,092 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.21 MB/102975732, heapSize ~128.00 MB/134218888, currentSize=469.54 KB/480812 for e5dc5c620be19b02666a19c280ff582e in 845ms, sequenceid=263061866, compaction requested=true 2020-11-17 05:40:27,093 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 05:40:27,094 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d365f50d37f24e1198282234d62dd603, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ebd97d1b8e6249bcb0f11b2ec59ed08a, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/15b816f3416843cb8e67417918713bb0] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=94.6 M 2020-11-17 05:40:27,122 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:40:27,124 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:40:27,145 INFO [Thread-1916] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:40:35,177 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:40:47,068 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#412 average throughput is 55.58 MB/second, slept 20 time(s) and total slept time is 7298 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 05:40:47,124 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:40:47,145 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 446cd1e6e4dc4f01a068e9e34231ad98(size=96.2 M), total size for store is 49.1 G. This selection was in queue for 0sec, and took 20sec to execute. 2020-11-17 05:40:47,145 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605591627092; duration=20sec 2020-11-17 05:41:09,308 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15604 ns. 2020-11-17 05:41:09,309 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 29994 ns. 2020-11-17 05:41:13,390 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591601303 with entries=27445, filesize=128.39 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591673352 2020-11-17 05:41:39,414 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 114064 ns. 2020-11-17 05:41:39,414 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 52778 ns. 2020-11-17 05:41:39,414 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 76477 ns. 2020-11-17 05:41:39,414 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 238239 ns. 2020-11-17 05:43:01,160 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:43:07,216 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591673352 with entries=41312, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591787185 2020-11-17 05:43:18,056 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:43:43,400 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:43:53,028 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:43:59,328 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.19 MB heapSize=128.78 MB 2020-11-17 05:43:59,391 INFO [Thread-1927] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:44:00,219 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.19 MB at sequenceid=263072533 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/c111528359004d7fa139cd328dc4edcf 2020-11-17 05:44:00,229 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:44:00,254 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c111528359004d7fa139cd328dc4edcf, entries=354794, sequenceid=263072533, filesize=8.1 M 2020-11-17 05:44:00,260 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.19 MB/108199418, heapSize ~128.78 MB/135038688, currentSize=1.56 MB/1633478 for e5dc5c620be19b02666a19c280ff582e in 932ms, sequenceid=263072533, compaction requested=true 2020-11-17 05:44:11,282 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.62 MB heapSize=128.36 MB 2020-11-17 05:44:11,322 INFO [Thread-1929] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:44:12,280 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.62 MB at sequenceid=251900561 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/2d2b96a771f04cfc85043388c27f745b 2020-11-17 05:44:12,287 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:44:12,308 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/2d2b96a771f04cfc85043388c27f745b, entries=310846, sequenceid=251900561, filesize=6.4 M 2020-11-17 05:44:12,309 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.62 MB/111800408, heapSize ~128.36 MB/134595280, currentSize=52.60 KB/53862 for 82421ce8c5c8b48c573baee83caa8305 in 1028ms, sequenceid=251900561, compaction requested=true 2020-11-17 05:44:15,686 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:44:17,161 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591787185 with entries=26119, filesize=128.62 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591857128 2020-11-17 05:44:26,766 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:44:32,019 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=174.98 MB, max=1.83 GB, blockCount=44133, accesses=18200798, hits=15447842, hitRatio=84.87%, , cachingAccesses=15499236, cachingHits=15443877, cachingHitsRatio=99.64%, evictions=2249, evicted=0, evictedPerRun=0.0 2020-11-17 05:44:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 63624 ns. 2020-11-17 05:44:40,559 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:44:43,893 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:44:52,934 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:45:09,060 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:45:15,054 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:45:29,954 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.98 MB heapSize=128.00 MB 2020-11-17 05:45:29,997 INFO [Thread-1934] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:45:30,759 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.98 MB at sequenceid=271874066 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/f4f4fa77e79c4ff2ba267b9542d03fd0 2020-11-17 05:45:30,774 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:45:30,808 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/f4f4fa77e79c4ff2ba267b9542d03fd0, entries=316542, sequenceid=271874066, filesize=8.7 M 2020-11-17 05:45:30,810 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.98 MB/110075089, heapSize ~128.00 MB/134217864, currentSize=24.49 KB/25082 for 0469ce862509756add4a8cdb00bc465c in 855ms, sequenceid=271874066, compaction requested=true 2020-11-17 05:45:35,517 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:45:45,019 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:45:54,063 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:11,599 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591857128 with entries=41880, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591971565 2020-11-17 05:46:18,708 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:18,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18254 ns. 2020-11-17 05:46:18,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 27895 ns. 2020-11-17 05:46:29,394 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.25 MB heapSize=128.25 MB 2020-11-17 05:46:29,454 INFO [Thread-1938] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:30,266 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.25 MB at sequenceid=259748686 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/4fff72a1a80643749c4d620968980257 2020-11-17 05:46:30,272 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:30,290 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/4fff72a1a80643749c4d620968980257, entries=373922, sequenceid=259748686, filesize=7.1 M 2020-11-17 05:46:30,293 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.25 MB/106167653, heapSize ~128.25 MB/134481648, currentSize=43.67 KB/44717 for 06f7ca9534710e37d90d9f8345864259 in 899ms, sequenceid=259748686, compaction requested=true 2020-11-17 05:46:30,294 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 05:46:30,294 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/be05cf584e9f41ad9239d65a52accdb6, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/8a8094eba4da4ecdaea58b821b8c7dad, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/28b36149ece54e1ebe807c2e6ea08e56, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/4fff72a1a80643749c4d620968980257] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=800.8 M 2020-11-17 05:46:30,373 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:30,385 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:30,405 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:30,405 INFO [Thread-1940] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:31,797 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:31,901 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:34,184 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:40,768 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:43,466 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:44,589 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:45,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:46,956 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:47,959 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:51,595 INFO [DataStreamer for file /hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/b998a62d0fb54340b22c6ffaac98300f] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:46:54,628 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:05,591 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:08,227 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.40 MB heapSize=128.10 MB 2020-11-17 05:47:08,308 INFO [Thread-1946] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:09,694 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.40 MB at sequenceid=261591587 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/1c854c8135a5468b8ca4d8fea1641afb 2020-11-17 05:47:09,720 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/1c854c8135a5468b8ca4d8fea1641afb, entries=576676, sequenceid=261591587, filesize=8.8 M 2020-11-17 05:47:09,722 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.40 MB/89551369, heapSize ~128.10 MB/134317944, currentSize=85.33 KB/87378 for 73aae5715542e1a5370f7540d2117abc in 1494ms, sequenceid=261591587, compaction requested=true 2020-11-17 05:47:12,911 INFO [DataStreamer for file /hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/b998a62d0fb54340b22c6ffaac98300f] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:18,991 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:21,892 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591971565 with entries=25931, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592041859 2020-11-17 05:47:25,660 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:29,248 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.51 MB heapSize=128.66 MB 2020-11-17 05:47:29,283 INFO [Thread-1949] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:30,114 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.51 MB at sequenceid=263083180 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/67395932a9674cdfad5665f59da53464 2020-11-17 05:47:30,122 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:30,147 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/67395932a9674cdfad5665f59da53464, entries=362613, sequenceid=263083180, filesize=8.1 M 2020-11-17 05:47:30,149 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.51 MB/107488683, heapSize ~128.66 MB/134914536, currentSize=2.05 MB/2145078 for e5dc5c620be19b02666a19c280ff582e in 901ms, sequenceid=263083180, compaction requested=true 2020-11-17 05:47:30,150 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 05:47:30,150 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/446cd1e6e4dc4f01a068e9e34231ad98, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c111528359004d7fa139cd328dc4edcf, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/67395932a9674cdfad5665f59da53464] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=112.5 M 2020-11-17 05:47:30,161 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:30,183 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:30,209 INFO [Thread-1951] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:32,011 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:38,162 INFO [DataStreamer for file /hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/b998a62d0fb54340b22c6ffaac98300f] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 154095 ns. 2020-11-17 05:47:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 47946 ns. 2020-11-17 05:47:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 250979 ns. 2020-11-17 05:47:38,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 128954 ns. 2020-11-17 05:47:41,372 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:47,568 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:54,185 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:47:55,065 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:00,204 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:03,625 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:03,627 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:03,629 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:03,633 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:16,333 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#420 average throughput is 28.47 MB/second, slept 25 time(s) and total slept time is 29523 ms. 1 active operations remaining, total limit is 57.69 MB/second 2020-11-17 05:48:16,411 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 58dfbba14d8c477da1c889db35c02dee(size=114.1 M), total size for store is 49.1 G. This selection was in queue for 0sec, and took 46sec to execute. 2020-11-17 05:48:16,412 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605592050150; duration=46sec 2020-11-17 05:48:23,196 INFO [DataStreamer for file /hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/b998a62d0fb54340b22c6ffaac98300f] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:31,817 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:32,523 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:33,714 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:33,966 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:33,969 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:33,971 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:34,182 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:34,184 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:34,827 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:35,360 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:35,687 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:35,689 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:35,813 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:35,935 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:35,938 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:37,408 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:37,564 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:38,275 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:40,014 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:40,182 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:40,185 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:40,187 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:40,322 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:40,326 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:42,950 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:42,952 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:42,954 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:46,328 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:46,330 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:46,332 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:46,334 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:46,336 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:46,338 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:46,465 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:46,987 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:46,989 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:47,113 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:47,362 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:49,188 INFO [DataStreamer for file /hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/b998a62d0fb54340b22c6ffaac98300f] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:48:51,930 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:49:03,871 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592041859 with entries=37917, filesize=128.09 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592143822 2020-11-17 05:49:04,396 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.61 MB heapSize=128.25 MB 2020-11-17 05:49:04,420 INFO [Thread-1961] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:49:05,061 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.61 MB at sequenceid=251909137 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/392bd188b9e94cd4a9a383a9751607ff 2020-11-17 05:49:05,067 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:49:05,085 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/392bd188b9e94cd4a9a383a9751607ff, entries=309543, sequenceid=251909137, filesize=6.3 M 2020-11-17 05:49:05,086 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.61 MB/111785367, heapSize ~128.25 MB/134479216, currentSize=194.84 KB/199515 for 82421ce8c5c8b48c573baee83caa8305 in 690ms, sequenceid=251909137, compaction requested=true 2020-11-17 05:49:05,087 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 05:49:05,087 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0a918c8b3c864fdf98102e1782bc4898, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/2d2b96a771f04cfc85043388c27f745b, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/392bd188b9e94cd4a9a383a9751607ff] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=57.9 M 2020-11-17 05:49:05,145 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:49:05,150 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:49:05,188 INFO [Thread-1963] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:49:27,961 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:49:27,964 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:49:28,115 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:49:34,793 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=169.40 MB, max=1.83 GB, blockCount=44249, accesses=19073820, hits=16149499, hitRatio=84.67%, , cachingAccesses=16200803, cachingHits=16145327, cachingHitsRatio=99.66%, evictions=2279, evicted=0, evictedPerRun=0.0 2020-11-17 05:49:38,254 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#422 average throughput is 28.83 MB/second, slept 19 time(s) and total slept time is 24289 ms. 1 active operations remaining, total limit is 57.69 MB/second 2020-11-17 05:49:38,318 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:49:38,341 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into d2e4e8506e1e44cfbf89e6a8fc082cde(size=57.8 M), total size for store is 11.1 G. This selection was in queue for 0sec, and took 33sec to execute. 2020-11-17 05:49:38,341 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605592145087; duration=33sec 2020-11-17 05:49:38,708 INFO [DataStreamer for file /hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/b998a62d0fb54340b22c6ffaac98300f] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:49:39,049 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 40258 ns. 2020-11-17 05:49:39,412 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:49:39,414 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:49:40,658 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#417 average throughput is 44.59 MB/second, slept 153 time(s) and total slept time is 85612 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 05:49:40,747 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into b998a62d0fb54340b22c6ffaac98300f(size=783.6 M), total size for store is 55.4 G. This selection was in queue for 0sec, and took 3mins, 10sec to execute. 2020-11-17 05:49:40,748 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=11, startTime=1605591990293; duration=3mins, 10sec 2020-11-17 05:50:19,545 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592143822 with entries=28655, filesize=128.24 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592219516 2020-11-17 05:50:39,099 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:51:18,974 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 16253 ns. 2020-11-17 05:51:18,974 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 30080 ns. 2020-11-17 05:52:15,083 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592219516 with entries=43133, filesize=128.22 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592335056 2020-11-17 05:52:21,477 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.48 MB heapSize=128.21 MB 2020-11-17 05:52:21,503 INFO [Thread-1974] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:52:22,294 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.48 MB at sequenceid=263098111 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/5ef6f1c80b7d4404bf7a160db19f71f5 2020-11-17 05:52:22,302 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:52:22,321 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5ef6f1c80b7d4404bf7a160db19f71f5, entries=413814, sequenceid=263098111, filesize=8.0 M 2020-11-17 05:52:22,325 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.48 MB/103268358, heapSize ~128.21 MB/134437968, currentSize=570.30 KB/583986 for e5dc5c620be19b02666a19c280ff582e in 849ms, sequenceid=263098111, compaction requested=true 2020-11-17 05:52:38,911 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 60934 ns. 2020-11-17 05:53:18,057 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:53:25,430 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592335056 with entries=26127, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592405402 2020-11-17 05:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 337287 ns. 2020-11-17 05:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 159377 ns. 2020-11-17 05:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 81760 ns. 2020-11-17 05:54:13,275 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.57 MB heapSize=128.00 MB 2020-11-17 05:54:13,307 INFO [Thread-1981] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:54:14,310 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.57 MB at sequenceid=287438333 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/721d2f27197d47ae8504890027cd0865 2020-11-17 05:54:14,317 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:54:14,332 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/721d2f27197d47ae8504890027cd0865, entries=523006, sequenceid=287438333, filesize=9.1 M 2020-11-17 05:54:14,335 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.57 MB/94969772, heapSize ~128.00 MB/134218080, currentSize=40.96 KB/41939 for 5e5f05205d7e40b0141cd977dd009ba1 in 1060ms, sequenceid=287438333, compaction requested=true 2020-11-17 05:54:35,253 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.19 MB heapSize=128.00 MB 2020-11-17 05:54:35,285 INFO [Thread-1984] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=180.03 MB, max=1.83 GB, blockCount=43974, accesses=19080092, hits=16151847, hitRatio=84.65%, , cachingAccesses=16203134, cachingHits=16147616, cachingHitsRatio=99.66%, evictions=2309, evicted=0, evictedPerRun=0.0 2020-11-17 05:54:36,339 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.19 MB at sequenceid=291307700 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/b5361f19a3894578a68e13fcb99c0d7e 2020-11-17 05:54:36,345 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:54:36,362 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/b5361f19a3894578a68e13fcb99c0d7e, entries=553007, sequenceid=291307700, filesize=8.9 M 2020-11-17 05:54:36,363 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~88.19 MB/92475283, heapSize ~128.00 MB/134220688, currentSize=46.83 KB/47950 for bdaa31a63b7f3d06d9909b7560333e79 in 1110ms, sequenceid=291307700, compaction requested=true 2020-11-17 05:54:38,220 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.36 MB heapSize=128.71 MB 2020-11-17 05:54:38,256 INFO [Thread-1986] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:54:38,980 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.36 MB at sequenceid=272908803 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/3f9679e24e5f4637a4f5a6339178bba7 2020-11-17 05:54:39,001 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/3f9679e24e5f4637a4f5a6339178bba7, entries=279371, sequenceid=272908803, filesize=8.5 M 2020-11-17 05:54:39,003 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.36 MB/113623680, heapSize ~128.71 MB/134964720, currentSize=690.91 KB/707488 for fc93f6dcc564341dcc983bf30971f73d in 783ms, sequenceid=272908803, compaction requested=true 2020-11-17 05:54:39,004 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 05:54:39,004 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/879c31fb7e1c48268c7e1d71dd97c1e5, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/058b704583dc4313b721c72e43e011dc, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/3f9679e24e5f4637a4f5a6339178bba7] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=25.5 M 2020-11-17 05:54:39,033 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:54:39,039 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:54:39,059 INFO [Thread-1988] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:54:44,474 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#427 average throughput is 59.70 MB/second, slept 6 time(s) and total slept time is 2816 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 05:54:44,546 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into e458796d94eb4e5e81291c596d90dde0(size=26.2 M), total size for store is 33.4 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 05:54:44,546 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=11, startTime=1605592479003; duration=5sec 2020-11-17 05:54:57,026 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.06 MB heapSize=128.02 MB 2020-11-17 05:54:57,059 INFO [Thread-1991] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:54:57,733 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.06 MB at sequenceid=251919643 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/c0f97090f21b4b1f83bb03afb1bc9544 2020-11-17 05:54:57,738 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:54:57,754 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/c0f97090f21b4b1f83bb03afb1bc9544, entries=313789, sequenceid=251919643, filesize=6.4 M 2020-11-17 05:54:57,756 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.06 MB/111212797, heapSize ~128.02 MB/134241928, currentSize=17.19 KB/17600 for 82421ce8c5c8b48c573baee83caa8305 in 730ms, sequenceid=251919643, compaction requested=true 2020-11-17 05:55:22,985 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592405402 with entries=43525, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592522951 2020-11-17 05:55:22,985 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590475769 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605590475769 2020-11-17 05:55:22,996 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590604659 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605590604659 2020-11-17 05:55:23,003 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590751942 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605590751942 2020-11-17 05:55:23,008 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590863808 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605590863808 2020-11-17 05:55:23,012 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605590948263 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605590948263 2020-11-17 05:55:23,017 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591059891 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605591059891 2020-11-17 05:55:23,022 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591126988 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605591126988 2020-11-17 05:55:23,027 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591238233 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605591238233 2020-11-17 05:55:23,034 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591311101 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605591311101 2020-11-17 05:55:23,039 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591424790 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605591424790 2020-11-17 05:55:23,043 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591494956 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605591494956 2020-11-17 05:55:23,048 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591601303 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605591601303 2020-11-17 05:55:23,053 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591673352 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605591673352 2020-11-17 05:55:23,058 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591787185 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605591787185 2020-11-17 05:55:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 38565 ns. 2020-11-17 05:55:53,495 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.52 MB heapSize=128.05 MB 2020-11-17 05:55:53,527 INFO [Thread-1996] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:55:54,289 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.52 MB at sequenceid=263108908 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/e19657d10c4445bb89570141e2238fae 2020-11-17 05:55:54,296 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:55:54,319 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e19657d10c4445bb89570141e2238fae, entries=382172, sequenceid=263108908, filesize=8.1 M 2020-11-17 05:55:54,320 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.52 MB/105406543, heapSize ~128.05 MB/134274168, currentSize=701.65 KB/718485 for e5dc5c620be19b02666a19c280ff582e in 826ms, sequenceid=263108908, compaction requested=true 2020-11-17 05:56:29,453 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 35569 ns. 2020-11-17 05:56:29,454 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 12547 ns. 2020-11-17 05:56:31,855 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592522951 with entries=25190, filesize=128.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592591824 2020-11-17 05:56:37,356 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:56:47,997 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:57:00,266 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:57:48,024 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.80 MB heapSize=128.00 MB 2020-11-17 05:57:48,072 INFO [Thread-2004] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:57:48,901 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.80 MB at sequenceid=259784061 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/5340b54cde8040e9b48f87a14a145c90 2020-11-17 05:57:48,908 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:57:48,939 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/5340b54cde8040e9b48f87a14a145c90, entries=375223, sequenceid=259784061, filesize=7.1 M 2020-11-17 05:57:48,942 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.80 MB/105701131, heapSize ~128.00 MB/134219632, currentSize=65.03 KB/66587 for 06f7ca9534710e37d90d9f8345864259 in 918ms, sequenceid=259784061, compaction requested=true 2020-11-17 05:57:57,235 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:58:24,950 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592591824 with entries=41882, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592704905 2020-11-17 05:58:32,799 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:58:38,918 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 61526 ns. 2020-11-17 05:59:02,617 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.99 MB heapSize=128.00 MB 2020-11-17 05:59:02,654 INFO [Thread-2010] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:59:03,359 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.99 MB at sequenceid=271913153 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/88b99c6542b84302a802ba96b7edb9de 2020-11-17 05:59:03,366 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:59:03,384 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/88b99c6542b84302a802ba96b7edb9de, entries=317042, sequenceid=271913153, filesize=8.7 M 2020-11-17 05:59:03,386 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.99 MB/110089063, heapSize ~128.00 MB/134219168, currentSize=38.23 KB/39152 for 0469ce862509756add4a8cdb00bc465c in 769ms, sequenceid=271913153, compaction requested=true 2020-11-17 05:59:03,387 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 05:59:03,387 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/ad5a26343b0c4aa19b0c0d74d5179935, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/f4f4fa77e79c4ff2ba267b9542d03fd0, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/88b99c6542b84302a802ba96b7edb9de] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=26.1 M 2020-11-17 05:59:03,423 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:59:03,427 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:59:03,452 INFO [Thread-2012] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:59:05,868 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:59:08,778 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#432 average throughput is 58.91 MB/second, slept 6 time(s) and total slept time is 2068 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 05:59:08,871 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into d7d6cfbd622148fd9b7340c32a7b3578(size=27.0 M), total size for store is 42.7 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 05:59:08,872 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605592743386; duration=5sec 2020-11-17 05:59:14,147 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.99 MB heapSize=128.47 MB 2020-11-17 05:59:14,187 INFO [Thread-2014] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:59:15,028 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.99 MB at sequenceid=263119151 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/9e588f1ccbee4dbabefa99fb2afca6a2 2020-11-17 05:59:15,063 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/9e588f1ccbee4dbabefa99fb2afca6a2, entries=353129, sequenceid=263119151, filesize=8.2 M 2020-11-17 05:59:15,064 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.99 MB/107989788, heapSize ~128.47 MB/134710472, currentSize=940.55 KB/963128 for e5dc5c620be19b02666a19c280ff582e in 917ms, sequenceid=263119151, compaction requested=true 2020-11-17 05:59:15,065 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 05:59:15,065 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5ef6f1c80b7d4404bf7a160db19f71f5, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e19657d10c4445bb89570141e2238fae, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/9e588f1ccbee4dbabefa99fb2afca6a2] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.3 M 2020-11-17 05:59:15,098 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:59:15,127 INFO [Thread-2016] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:59:16,385 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:59:20,428 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#434 average throughput is 56.79 MB/second, slept 5 time(s) and total slept time is 973 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 05:59:20,542 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into c6815e783a524354a772117bea47bf10(size=25.3 M), total size for store is 49.1 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 05:59:20,542 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605592755065; duration=5sec 2020-11-17 05:59:24,581 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 05:59:36,027 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=178.60 MB, max=1.83 GB, blockCount=44006, accesses=19243593, hits=16300101, hitRatio=84.70%, , cachingAccesses=16351402, cachingHits=16295843, cachingHitsRatio=99.66%, evictions=2339, evicted=0, evictedPerRun=0.0 2020-11-17 05:59:36,329 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592704905 with entries=25989, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592776276 2020-11-17 05:59:36,329 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591857128 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605591857128 2020-11-17 05:59:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 176462 ns. 2020-11-17 05:59:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 267973 ns. 2020-11-17 05:59:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 125627 ns. 2020-11-17 06:00:17,438 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.38 MB heapSize=128.22 MB 2020-11-17 06:00:17,483 INFO [Thread-2021] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:00:18,118 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.38 MB at sequenceid=251929077 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/9d65f41a69884012a216a6eddfd0aa8d 2020-11-17 06:00:18,124 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:00:18,148 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/9d65f41a69884012a216a6eddfd0aa8d, entries=312180, sequenceid=251929077, filesize=6.3 M 2020-11-17 06:00:18,151 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.38 MB/111546619, heapSize ~128.22 MB/134447288, currentSize=245.67 KB/251571 for 82421ce8c5c8b48c573baee83caa8305 in 714ms, sequenceid=251929077, compaction requested=true 2020-11-17 06:00:18,152 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 06:00:18,157 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d2e4e8506e1e44cfbf89e6a8fc082cde, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/c0f97090f21b4b1f83bb03afb1bc9544, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/9d65f41a69884012a216a6eddfd0aa8d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=70.5 M 2020-11-17 06:00:18,197 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:00:18,202 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:00:18,240 INFO [Thread-2023] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:00:38,493 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#436 average throughput is 57.50 MB/second, slept 23 time(s) and total slept time is 9749 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:00:38,559 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 36550f897a334228a8bca8d2e1f5b28f(size=70.4 M), total size for store is 11.1 G. This selection was in queue for 0sec, and took 20sec to execute. 2020-11-17 06:00:38,559 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605592818151; duration=20sec 2020-11-17 06:00:38,756 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 60474 ns. 2020-11-17 06:01:21,673 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:01:31,328 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592776276 with entries=42879, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592891273 2020-11-17 06:01:39,264 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 11610 ns. 2020-11-17 06:01:39,265 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 33642 ns. 2020-11-17 06:02:02,500 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:02:05,998 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:02:41,194 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592891273 with entries=25701, filesize=128.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592961166 2020-11-17 06:03:18,067 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:03:21,362 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:03:42,002 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:03:52,138 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.32 MB heapSize=128.06 MB 2020-11-17 06:03:52,183 INFO [Thread-2034] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:03:52,955 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.32 MB at sequenceid=263132996 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/42a8accd7d234a92a3036a2aabecd395 2020-11-17 06:03:52,962 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:03:52,982 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/42a8accd7d234a92a3036a2aabecd395, entries=371260, sequenceid=263132996, filesize=8.2 M 2020-11-17 06:03:52,983 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.32 MB/106240878, heapSize ~128.06 MB/134283584, currentSize=202.94 KB/207810 for e5dc5c620be19b02666a19c280ff582e in 845ms, sequenceid=263132996, compaction requested=true 2020-11-17 06:04:02,446 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=179.73 MB, max=1.83 GB, blockCount=43981, accesses=19390913, hits=16428512, hitRatio=84.72%, , cachingAccesses=16479824, cachingHits=16424254, cachingHitsRatio=99.66%, evictions=2369, evicted=0, evictedPerRun=0.0 2020-11-17 06:04:37,092 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.74 MB heapSize=128.09 MB 2020-11-17 06:04:37,126 INFO [Thread-2038] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:04:37,580 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592961166 with entries=42302, filesize=128.62 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593077559 2020-11-17 06:04:38,250 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.74 MB at sequenceid=261646166 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/857a6582472c43c89c24f0943abdeea3 2020-11-17 06:04:38,285 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:04:38,314 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/857a6582472c43c89c24f0943abdeea3, entries=581982, sequenceid=261646166, filesize=9.1 M 2020-11-17 06:04:38,319 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.74 MB/89905264, heapSize ~128.09 MB/134307656, currentSize=284.52 KB/291344 for 73aae5715542e1a5370f7540d2117abc in 1227ms, sequenceid=261646166, compaction requested=true 2020-11-17 06:04:38,320 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 06:04:38,321 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/2af08ab4a1d94819bcc817ad9dbd4a2d, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/1c854c8135a5468b8ca4d8fea1641afb, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/857a6582472c43c89c24f0943abdeea3] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=118.3 M 2020-11-17 06:04:38,361 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:04:38,365 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:04:38,506 INFO [Thread-2040] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:04:39,243 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 59693 ns. 2020-11-17 06:04:53,139 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:04:57,263 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:04:57,436 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#439 average throughput is 56.17 MB/second, slept 15 time(s) and total slept time is 3086 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:04:57,589 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 35c7c16016f2485cb35e87d6bd68e5f0(size=118.5 M), total size for store is 12.5 G. This selection was in queue for 0sec, and took 19sec to execute. 2020-11-17 06:04:57,589 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605593078320; duration=19sec 2020-11-17 06:05:09,280 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:05:39,654 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 211400 ns. 2020-11-17 06:05:39,654 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 178223 ns. 2020-11-17 06:05:39,654 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 156994 ns. 2020-11-17 06:05:39,654 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 46383 ns. 2020-11-17 06:05:42,424 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:05:54,389 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593077559 with entries=28084, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593154349 2020-11-17 06:05:54,389 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605591971565 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605591971565 2020-11-17 06:05:54,397 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592041859 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605592041859 2020-11-17 06:05:54,405 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592143822 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605592143822 2020-11-17 06:05:54,411 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592219516 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605592219516 2020-11-17 06:05:54,419 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592335056 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605592335056 2020-11-17 06:06:32,258 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.01 MB heapSize=128.00 MB 2020-11-17 06:06:32,294 INFO [Thread-2047] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:06:32,910 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.01 MB at sequenceid=251940027 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/ca35a5673e214f79ac144a16271b4edc 2020-11-17 06:06:32,928 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:06:32,961 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/ca35a5673e214f79ac144a16271b4edc, entries=314230, sequenceid=251940027, filesize=6.4 M 2020-11-17 06:06:32,962 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.01 MB/111156271, heapSize ~128.00 MB/134218064, currentSize=21.42 KB/21933 for 82421ce8c5c8b48c573baee83caa8305 in 704ms, sequenceid=251940027, compaction requested=true 2020-11-17 06:06:48,763 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 9984 ns. 2020-11-17 06:06:48,763 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 27512 ns. 2020-11-17 06:07:37,694 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.09 MB heapSize=128.09 MB 2020-11-17 06:07:37,726 INFO [Thread-2052] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:07:38,461 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.09 MB at sequenceid=263144344 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/d4f8f8d098ac40e3bbeedeaba9781bcc 2020-11-17 06:07:38,468 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:07:38,494 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d4f8f8d098ac40e3bbeedeaba9781bcc, entries=417695, sequenceid=263144344, filesize=8.0 M 2020-11-17 06:07:38,500 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~98.09 MB/102851203, heapSize ~128.09 MB/134307776, currentSize=224.74 KB/230135 for e5dc5c620be19b02666a19c280ff582e in 807ms, sequenceid=263144344, compaction requested=true 2020-11-17 06:07:38,501 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 06:07:38,502 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c6815e783a524354a772117bea47bf10, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/42a8accd7d234a92a3036a2aabecd395, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d4f8f8d098ac40e3bbeedeaba9781bcc] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.5 M 2020-11-17 06:07:38,529 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:07:38,531 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:07:38,553 INFO [Thread-2054] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:07:46,788 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#442 average throughput is 60.77 MB/second, slept 9 time(s) and total slept time is 2352 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 06:07:46,859 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 7817d1fa456c43d6925c147d9aa35985(size=42.8 M), total size for store is 49.1 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 06:07:46,860 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605593258501; duration=8sec 2020-11-17 06:07:57,922 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593154349 with entries=45554, filesize=128.14 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593277875 2020-11-17 06:08:19,467 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=181.17 MB, max=1.83 GB, blockCount=43941, accesses=19419316, hits=16431216, hitRatio=84.61%, , cachingAccesses=16482492, cachingHits=16426909, cachingHitsRatio=99.66%, evictions=2399, evicted=0, evictedPerRun=0.0 2020-11-17 06:09:37,157 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593277875 with entries=37834, filesize=128.30 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593377127 2020-11-17 06:09:52,549 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.73 MB heapSize=128.00 MB 2020-11-17 06:09:52,596 INFO [Thread-2061] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:09:53,493 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.73 MB at sequenceid=272936503 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/60d9599aa22840a8a5b714e19244b05f 2020-11-17 06:09:53,498 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:09:53,515 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/60d9599aa22840a8a5b714e19244b05f, entries=278197, sequenceid=272936503, filesize=8.5 M 2020-11-17 06:09:53,516 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~107.73 MB/112965584, heapSize ~128.00 MB/134217736, currentSize=27.06 KB/27707 for fc93f6dcc564341dcc983bf30971f73d in 967ms, sequenceid=272936503, compaction requested=true 2020-11-17 06:10:20,339 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=99.96 MB heapSize=128.02 MB 2020-11-17 06:10:20,508 INFO [Thread-2065] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:10:21,239 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.96 MB at sequenceid=259823194 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/9985820f19fb416986b27d67e75b226c 2020-11-17 06:10:21,245 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:10:21,262 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/9985820f19fb416986b27d67e75b226c, entries=385256, sequenceid=259823194, filesize=7.3 M 2020-11-17 06:10:21,265 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.96 MB/104816411, heapSize ~128.02 MB/134235848, currentSize=30.73 KB/31467 for 06f7ca9534710e37d90d9f8345864259 in 925ms, sequenceid=259823194, compaction requested=true 2020-11-17 06:10:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 53885 ns. 2020-11-17 06:11:07,371 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.89 MB heapSize=128.00 MB 2020-11-17 06:11:07,406 INFO [Thread-2069] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:11:08,228 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.89 MB at sequenceid=263155033 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a92e4fcfebb84635a60045831aa9200d 2020-11-17 06:11:08,233 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:11:08,252 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a92e4fcfebb84635a60045831aa9200d, entries=362273, sequenceid=263155033, filesize=8.2 M 2020-11-17 06:11:08,254 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.89 MB/106836049, heapSize ~128.00 MB/134219416, currentSize=1.10 MB/1154993 for e5dc5c620be19b02666a19c280ff582e in 883ms, sequenceid=263155033, compaction requested=true 2020-11-17 06:11:15,413 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593377127 with entries=36175, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593475389 2020-11-17 06:11:39,009 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 45574 ns. 2020-11-17 06:11:39,009 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 264946 ns. 2020-11-17 06:11:39,009 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 122126 ns. 2020-11-17 06:11:39,009 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 131105 ns. 2020-11-17 06:11:49,490 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 27547 ns. 2020-11-17 06:11:49,490 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 9717 ns. 2020-11-17 06:12:27,082 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:12:42,690 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.76 MB heapSize=128.00 MB 2020-11-17 06:12:42,719 INFO [Thread-2075] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:12:43,434 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.76 MB at sequenceid=271952991 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/b4c1dd1108624b6b8d07b779f142a01e 2020-11-17 06:12:43,440 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:12:43,459 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/b4c1dd1108624b6b8d07b779f142a01e, entries=319589, sequenceid=271952991, filesize=8.6 M 2020-11-17 06:12:43,460 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.76 MB/109847804, heapSize ~128.00 MB/134218544, currentSize=22.42 KB/22962 for 0469ce862509756add4a8cdb00bc465c in 770ms, sequenceid=271952991, compaction requested=true 2020-11-17 06:13:00,658 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:13:08,706 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593475389 with entries=40098, filesize=128.19 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593588681 2020-11-17 06:13:16,212 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:13:32,864 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:13:41,097 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:13:46,303 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:13:59,454 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:14:12,253 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:14:18,283 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.76 MB heapSize=128.16 MB 2020-11-17 06:14:18,338 INFO [Thread-2081] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:14:19,002 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.76 MB at sequenceid=251953223 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/8684554d2b1f44658881676e073ab8d7 2020-11-17 06:14:19,027 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/8684554d2b1f44658881676e073ab8d7, entries=319468, sequenceid=251953223, filesize=6.6 M 2020-11-17 06:14:19,028 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.76 MB/110897943, heapSize ~128.16 MB/134383952, currentSize=697.02 KB/713753 for 82421ce8c5c8b48c573baee83caa8305 in 745ms, sequenceid=251953223, compaction requested=true 2020-11-17 06:14:19,029 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 06:14:19,030 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/36550f897a334228a8bca8d2e1f5b28f, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/ca35a5673e214f79ac144a16271b4edc, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/8684554d2b1f44658881676e073ab8d7] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=83.4 M 2020-11-17 06:14:19,059 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:14:19,061 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:14:19,088 INFO [Thread-2083] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:14:32,151 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=180.11 MB, max=1.83 GB, blockCount=43965, accesses=19446013, hits=16441683, hitRatio=84.55%, , cachingAccesses=16492994, cachingHits=16437376, cachingHitsRatio=99.66%, evictions=2429, evicted=0, evictedPerRun=0.0 2020-11-17 06:14:40,048 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:14:40,199 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:14:41,955 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:14:42,890 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#448 average throughput is 57.76 MB/second, slept 27 time(s) and total slept time is 11132 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:14:42,984 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:14:43,009 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into f0c1d88f56a043cd9c21ac7756575fce(size=83.2 M), total size for store is 11.2 G. This selection was in queue for 0sec, and took 23sec to execute. 2020-11-17 06:14:43,009 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605593659029; duration=23sec 2020-11-17 06:14:57,491 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:14:59,297 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593588681 with entries=31243, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593699257 2020-11-17 06:15:10,371 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:15:37,612 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.93 MB heapSize=128.74 MB 2020-11-17 06:15:37,680 INFO [Thread-2088] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:15:38,550 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.93 MB at sequenceid=263167004 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/bf1c3a594b0b4e66a64ed050bf0cd7ab 2020-11-17 06:15:38,558 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:15:38,575 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/bf1c3a594b0b4e66a64ed050bf0cd7ab, entries=358010, sequenceid=263167004, filesize=7.8 M 2020-11-17 06:15:38,578 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.93 MB/107925358, heapSize ~128.74 MB/134988680, currentSize=1.86 MB/1947132 for e5dc5c620be19b02666a19c280ff582e in 966ms, sequenceid=263167004, compaction requested=true 2020-11-17 06:15:38,579 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 06:15:38,579 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7817d1fa456c43d6925c147d9aa35985, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a92e4fcfebb84635a60045831aa9200d, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/bf1c3a594b0b4e66a64ed050bf0cd7ab] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=58.8 M 2020-11-17 06:15:38,608 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:15:38,610 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:15:38,644 INFO [Thread-2090] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:15:38,911 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 52528 ns. 2020-11-17 06:15:50,201 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593699257 with entries=15238, filesize=128.47 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593750055 2020-11-17 06:15:50,319 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#450 average throughput is 60.38 MB/second, slept 13 time(s) and total slept time is 3618 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 06:15:50,411 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into a7914eade09a4961bafbd4723f42d1ee(size=59.9 M), total size for store is 49.1 G. This selection was in queue for 0sec, and took 11sec to execute. 2020-11-17 06:15:50,411 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605593738578; duration=11sec 2020-11-17 06:16:10,147 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:16:39,334 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 45240 ns. 2020-11-17 06:16:49,085 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:16:58,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 16357 ns. 2020-11-17 06:16:58,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 25784 ns. 2020-11-17 06:17:00,016 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:17:03,369 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593750055 with entries=21999, filesize=128.76 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593823309 2020-11-17 06:17:38,718 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 345579 ns. 2020-11-17 06:17:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 72441 ns. 2020-11-17 06:17:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 61319 ns. 2020-11-17 06:18:01,594 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:07,433 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.27 MB heapSize=128.09 MB 2020-11-17 06:18:07,463 INFO [Thread-2099] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:08,389 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.27 MB at sequenceid=263173122 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/b15baa2dadc04bbbbf5656acb2e140d8 2020-11-17 06:18:08,433 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b15baa2dadc04bbbbf5656acb2e140d8, entries=357869, sequenceid=263173122, filesize=7.0 M 2020-11-17 06:18:08,435 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.27 MB/107232800, heapSize ~128.09 MB/134313208, currentSize=997.28 KB/1021214 for e5dc5c620be19b02666a19c280ff582e in 1002ms, sequenceid=263173122, compaction requested=true 2020-11-17 06:18:15,559 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:15,846 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.16 MB heapSize=128.00 MB 2020-11-17 06:18:15,875 INFO [Thread-2101] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:16,906 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.16 MB at sequenceid=287513228 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/8dc0222bff2948e7bd4d277b930423e5 2020-11-17 06:18:16,937 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/8dc0222bff2948e7bd4d277b930423e5, entries=528620, sequenceid=287513228, filesize=9.1 M 2020-11-17 06:18:16,941 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.16 MB/94543508, heapSize ~128.00 MB/134218296, currentSize=27.36 KB/28018 for 5e5f05205d7e40b0141cd977dd009ba1 in 1095ms, sequenceid=287513228, compaction requested=true 2020-11-17 06:18:16,941 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 06:18:16,941 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/d2e0233766f74cc084f39b4eadd1eb59, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/721d2f27197d47ae8504890027cd0865, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/8dc0222bff2948e7bd4d277b930423e5] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=102.4 M 2020-11-17 06:18:16,951 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:16,989 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:17,013 INFO [Thread-2103] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:17,155 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:22,654 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:24,951 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:30,485 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:30,486 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:30,627 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:30,629 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:31,608 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:32,121 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593823309 with entries=27119, filesize=128.72 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593912067 2020-11-17 06:18:32,735 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:33,076 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:34,405 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:34,448 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#453 average throughput is 55.04 MB/second, slept 15 time(s) and total slept time is 4496 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:18:34,522 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into 9c752d45038d4f05ac947dccd82e4630(size=102.7 M), total size for store is 24.3 G. This selection was in queue for 0sec, and took 17sec to execute. 2020-11-17 06:18:34,522 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=11, startTime=1605593896941; duration=17sec 2020-11-17 06:18:45,399 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:18:48,437 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:19:19,295 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:19:22,738 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:19:26,760 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:19:33,238 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=175.61 MB, max=1.83 GB, blockCount=44069, accesses=19982448, hits=16944678, hitRatio=84.80%, , cachingAccesses=16996082, cachingHits=16940348, cachingHitsRatio=99.67%, evictions=2459, evicted=0, evictedPerRun=0.0 2020-11-17 06:19:47,530 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:20:01,123 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:20:03,164 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.89 MB heapSize=128.00 MB 2020-11-17 06:20:03,221 INFO [Thread-2109] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:20:04,321 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.89 MB at sequenceid=261690381 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/5df2c588d5014764a66a5922767e70cf 2020-11-17 06:20:04,327 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:20:04,348 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/5df2c588d5014764a66a5922767e70cf, entries=575414, sequenceid=261690381, filesize=8.5 M 2020-11-17 06:20:04,349 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.89 MB/90059189, heapSize ~128.00 MB/134217768, currentSize=67.92 KB/69548 for 73aae5715542e1a5370f7540d2117abc in 1185ms, sequenceid=261690381, compaction requested=true 2020-11-17 06:20:16,032 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:20:39,031 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 52293 ns. 2020-11-17 06:20:41,772 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593912067 with entries=40265, filesize=128.17 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594041731 2020-11-17 06:20:48,148 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:20:52,946 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:20:56,019 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:21:07,974 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:21:12,442 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.94 MB heapSize=128.14 MB 2020-11-17 06:21:12,476 INFO [Thread-2113] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:21:13,264 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.94 MB at sequenceid=263181082 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/1c98a18d8c1f46eea141b60e8626754e 2020-11-17 06:21:13,271 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:21:13,287 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1c98a18d8c1f46eea141b60e8626754e, entries=349490, sequenceid=263181082, filesize=8.2 M 2020-11-17 06:21:13,290 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.94 MB/107935635, heapSize ~128.14 MB/134361744, currentSize=770.13 KB/788613 for e5dc5c620be19b02666a19c280ff582e in 847ms, sequenceid=263181082, compaction requested=true 2020-11-17 06:21:13,291 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 06:21:13,292 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a7914eade09a4961bafbd4723f42d1ee, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b15baa2dadc04bbbbf5656acb2e140d8, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1c98a18d8c1f46eea141b60e8626754e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=75.1 M 2020-11-17 06:21:13,322 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:21:13,324 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:21:13,351 INFO [Thread-2115] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:21:14,931 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:21:29,559 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#456 average throughput is 56.16 MB/second, slept 17 time(s) and total slept time is 6712 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:21:29,647 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 7196cf7263444f379980848f2e7ed2f0(size=75.9 M), total size for store is 49.1 G. This selection was in queue for 0sec, and took 16sec to execute. 2020-11-17 06:21:29,647 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605594073291; duration=16sec 2020-11-17 06:21:59,593 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 25118 ns. 2020-11-17 06:21:59,593 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18237 ns. 2020-11-17 06:22:38,582 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:22:38,776 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 52726 ns. 2020-11-17 06:22:54,657 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594041731 with entries=42453, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594174628 2020-11-17 06:23:14,762 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:23:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 314998 ns. 2020-11-17 06:23:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 78725 ns. 2020-11-17 06:23:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 73458 ns. 2020-11-17 06:23:57,995 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:24:24,905 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:24:31,435 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:24:36,025 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=174.00 MB, max=1.83 GB, blockCount=44166, accesses=20460128, hits=17407413, hitRatio=85.08%, , cachingAccesses=17458955, cachingHits=17403066, cachingHitsRatio=99.68%, evictions=2489, evicted=0, evictedPerRun=0.0 2020-11-17 06:24:56,049 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594174628 with entries=38265, filesize=128.56 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594296003 2020-11-17 06:24:56,069 WARN [Close-WAL-Writer-239] asyncfs.FanOutOneBlockAsyncDFSOutputHelper: complete file /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594174628 not finished, retry = 0 2020-11-17 06:25:25,956 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.15 MB heapSize=128.33 MB 2020-11-17 06:25:25,986 INFO [Thread-2129] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:25:26,695 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.15 MB at sequenceid=272958417 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/c2bce38f43074e71947d4b342eb814c8 2020-11-17 06:25:26,705 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:25:26,724 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/c2bce38f43074e71947d4b342eb814c8, entries=276767, sequenceid=272958417, filesize=8.5 M 2020-11-17 06:25:26,726 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.15 MB/113400668, heapSize ~128.33 MB/134563328, currentSize=19.09 KB/19544 for fc93f6dcc564341dcc983bf30971f73d in 771ms, sequenceid=272958417, compaction requested=true 2020-11-17 06:25:26,728 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 06:25:26,728 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/e458796d94eb4e5e81291c596d90dde0, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/60d9599aa22840a8a5b714e19244b05f, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/c2bce38f43074e71947d4b342eb814c8] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=43.1 M 2020-11-17 06:25:26,760 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:25:26,764 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:25:26,787 INFO [Thread-2131] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:25:35,963 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#458 average throughput is 58.72 MB/second, slept 9 time(s) and total slept time is 3471 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:25:36,039 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into cf40bb813d3f48b1921ee1644a017db9(size=43.7 M), total size for store is 33.4 G. This selection was in queue for 0sec, and took 9sec to execute. 2020-11-17 06:25:36,039 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=11, startTime=1605594326726; duration=9sec 2020-11-17 06:25:37,379 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=99.72 MB heapSize=128.19 MB 2020-11-17 06:25:37,409 INFO [Thread-2133] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:25:38,266 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.72 MB at sequenceid=259864723 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/c2a192ae1545465eb056ca2312808edb 2020-11-17 06:25:38,288 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/c2a192ae1545465eb056ca2312808edb, entries=395200, sequenceid=259864723, filesize=7.6 M 2020-11-17 06:25:38,289 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~99.72 MB/104558993, heapSize ~128.19 MB/134416472, currentSize=29.62 KB/30330 for 06f7ca9534710e37d90d9f8345864259 in 910ms, sequenceid=259864723, compaction requested=true 2020-11-17 06:25:38,290 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 06:25:38,290 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/5340b54cde8040e9b48f87a14a145c90, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/9985820f19fb416986b27d67e75b226c, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/c2a192ae1545465eb056ca2312808edb] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=21.9 M 2020-11-17 06:25:38,342 INFO [Thread-2135] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:25:43,082 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#460 average throughput is 62.73 MB/second, slept 5 time(s) and total slept time is 1866 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:25:43,143 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 70b061d3fd0d4ef5b609e85df0fba5ae(size=22.1 M), total size for store is 55.4 G. This selection was in queue for 0sec, and took 4sec to execute. 2020-11-17 06:25:43,144 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=11, startTime=1605594338289; duration=4sec 2020-11-17 06:25:52,513 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.43 MB heapSize=128.19 MB 2020-11-17 06:25:52,540 INFO [Thread-2137] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:25:53,357 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.43 MB at sequenceid=263193226 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/886b5d6fcad9476dad653229222f78b5 2020-11-17 06:25:53,363 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:25:53,379 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/886b5d6fcad9476dad653229222f78b5, entries=386404, sequenceid=263193226, filesize=8.1 M 2020-11-17 06:25:53,380 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.43 MB/105305254, heapSize ~128.19 MB/134414720, currentSize=231.34 KB/236894 for e5dc5c620be19b02666a19c280ff582e in 867ms, sequenceid=263193226, compaction requested=true 2020-11-17 06:26:05,221 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.05 MB heapSize=128.00 MB 2020-11-17 06:26:05,268 INFO [Thread-2140] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:26:05,939 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.05 MB at sequenceid=271986790 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/ffd370d1c4be402dbf111116fa65b1fb 2020-11-17 06:26:05,945 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:26:05,961 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/ffd370d1c4be402dbf111116fa65b1fb, entries=315355, sequenceid=271986790, filesize=8.7 M 2020-11-17 06:26:05,963 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.05 MB/110152851, heapSize ~128.00 MB/134220608, currentSize=16.65 KB/17046 for 0469ce862509756add4a8cdb00bc465c in 742ms, sequenceid=271986790, compaction requested=true 2020-11-17 06:26:05,963 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 06:26:05,964 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/d7d6cfbd622148fd9b7340c32a7b3578, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/b4c1dd1108624b6b8d07b779f142a01e, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/ffd370d1c4be402dbf111116fa65b1fb] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=44.3 M 2020-11-17 06:26:05,990 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:26:05,994 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:26:06,013 INFO [Thread-2142] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:26:15,466 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#463 average throughput is 55.32 MB/second, slept 5 time(s) and total slept time is 2073 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:26:15,536 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into e5cef750c77e4b61a081a737d4e259cf(size=45.2 M), total size for store is 42.8 G. This selection was in queue for 0sec, and took 9sec to execute. 2020-11-17 06:26:15,536 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605594365963; duration=9sec 2020-11-17 06:26:19,634 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.00 MB heapSize=128.00 MB 2020-11-17 06:26:19,656 INFO [Thread-2144] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:26:19,787 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:26:20,461 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.00 MB at sequenceid=251968982 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/1f01dbad25ae420184aa2f30ef838f2f 2020-11-17 06:26:20,487 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/1f01dbad25ae420184aa2f30ef838f2f, entries=327718, sequenceid=251968982, filesize=6.7 M 2020-11-17 06:26:20,488 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.00 MB/110095859, heapSize ~128.00 MB/134219040, currentSize=15.39 KB/15760 for 82421ce8c5c8b48c573baee83caa8305 in 854ms, sequenceid=251968982, compaction requested=true 2020-11-17 06:26:29,860 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.35 MB heapSize=128.00 MB 2020-11-17 06:26:29,888 INFO [Thread-2146] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:26:30,977 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.35 MB at sequenceid=291401451 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/0a1970e0060f40cea26fff2dd021295d 2020-11-17 06:26:30,982 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:26:31,003 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/0a1970e0060f40cea26fff2dd021295d, entries=550868, sequenceid=291401451, filesize=8.9 M 2020-11-17 06:26:31,004 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~88.35 MB/92644843, heapSize ~128.00 MB/134219136, currentSize=29.47 KB/30177 for bdaa31a63b7f3d06d9909b7560333e79 in 1144ms, sequenceid=291401451, compaction requested=true 2020-11-17 06:26:31,005 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-17 06:26:31,005 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/992eaf481d8243a49134bf2faa07c945, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/b5361f19a3894578a68e13fcb99c0d7e, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/0a1970e0060f40cea26fff2dd021295d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp, totalSize=26.7 M 2020-11-17 06:26:31,037 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:26:31,042 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:26:31,065 INFO [Thread-2148] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:26:34,812 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:26:35,550 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: bdaa31a63b7f3d06d9909b7560333e79#e#compaction#466 average throughput is 58.30 MB/second, slept 4 time(s) and total slept time is 963 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:26:35,640 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of bdaa31a63b7f3d06d9909b7560333e79 into f2ccc1a9025d488bb6ae6ef9e91d33a8(size=27.1 M), total size for store is 31.8 G. This selection was in queue for 0sec, and took 4sec to execute. 2020-11-17 06:26:35,640 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., storeName=e, priority=10, startTime=1605594391005; duration=4sec 2020-11-17 06:26:38,679 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 69536 ns. 2020-11-17 06:26:48,006 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:26:58,303 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594296003 with entries=38156, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594418260 2020-11-17 06:26:58,303 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592405402 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605592405402 2020-11-17 06:26:58,312 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592522951 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605592522951 2020-11-17 06:26:58,319 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592591824 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605592591824 2020-11-17 06:26:58,324 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592704905 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605592704905 2020-11-17 06:26:58,329 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592776276 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605592776276 2020-11-17 06:26:58,333 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592891273 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605592891273 2020-11-17 06:26:58,338 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605592961166 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605592961166 2020-11-17 06:26:58,342 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593077559 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605593077559 2020-11-17 06:26:58,346 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593154349 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605593154349 2020-11-17 06:26:58,351 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593277875 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605593277875 2020-11-17 06:26:58,356 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593377127 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605593377127 2020-11-17 06:26:58,361 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593475389 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605593475389 2020-11-17 06:26:58,367 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593588681 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605593588681 2020-11-17 06:26:58,371 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593699257 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605593699257 2020-11-17 06:26:58,376 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593750055 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605593750055 2020-11-17 06:27:08,725 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 10648 ns. 2020-11-17 06:27:08,725 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 30474 ns. 2020-11-17 06:28:07,245 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:28:39,546 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 52128 ns. 2020-11-17 06:29:23,934 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=175.75 MB, max=1.83 GB, blockCount=44114, accesses=20490592, hits=17411547, hitRatio=84.97%, , cachingAccesses=17463054, cachingHits=17407149, cachingHitsRatio=99.68%, evictions=2519, evicted=0, evictedPerRun=0.0 2020-11-17 06:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 06:29:39,104 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 86708 ns. 2020-11-17 06:29:39,105 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 111826 ns. 2020-11-17 06:29:39,105 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 303809 ns. 2020-11-17 06:29:39,749 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594418260 with entries=49311, filesize=128.15 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594579707 2020-11-17 06:29:49,196 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.00 MB heapSize=128.23 MB 2020-11-17 06:29:49,222 INFO [Thread-2157] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:29:50,019 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.00 MB at sequenceid=263203139 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/c9f9c9ef93974d99ba26b0a4a301e01e 2020-11-17 06:29:50,034 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:29:50,104 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c9f9c9ef93974d99ba26b0a4a301e01e, entries=378460, sequenceid=263203139, filesize=8.2 M 2020-11-17 06:29:50,118 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.00 MB/105908355, heapSize ~128.23 MB/134457744, currentSize=563.91 KB/577446 for e5dc5c620be19b02666a19c280ff582e in 920ms, sequenceid=263203139, compaction requested=true 2020-11-17 06:29:50,120 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 06:29:50,120 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7196cf7263444f379980848f2e7ed2f0, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/886b5d6fcad9476dad653229222f78b5, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c9f9c9ef93974d99ba26b0a4a301e01e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=92.2 M 2020-11-17 06:29:50,213 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:29:50,225 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:29:50,258 INFO [Thread-2159] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:29:53,329 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:30:01,111 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:30:09,568 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#468 average throughput is 57.56 MB/second, slept 21 time(s) and total slept time is 7027 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:30:09,656 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 038d57c4403746228cd2c3e75ab1d545(size=93.7 M), total size for store is 49.2 G. This selection was in queue for 0sec, and took 19sec to execute. 2020-11-17 06:30:09,656 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605594590118; duration=19sec 2020-11-17 06:30:13,813 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:31:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 56175 ns. 2020-11-17 06:31:43,495 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594579707 with entries=38352, filesize=128.51 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594703473 2020-11-17 06:32:01,859 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:32:08,834 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 13743 ns. 2020-11-17 06:32:08,834 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 32060 ns. 2020-11-17 06:33:17,648 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.57 MB heapSize=128.35 MB 2020-11-17 06:33:17,689 INFO [Thread-2169] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:33:18,372 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.57 MB at sequenceid=263212002 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/709fd087408945bf8f4c98346881620a 2020-11-17 06:33:18,379 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:33:18,402 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/709fd087408945bf8f4c98346881620a, entries=357780, sequenceid=263212002, filesize=8.1 M 2020-11-17 06:33:18,403 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.57 MB/107553686, heapSize ~128.35 MB/134583160, currentSize=335.84 KB/343903 for e5dc5c620be19b02666a19c280ff582e in 756ms, sequenceid=263212002, compaction requested=true 2020-11-17 06:33:18,405 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 06:33:18,405 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/58dfbba14d8c477da1c889db35c02dee, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/038d57c4403746228cd2c3e75ab1d545, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/709fd087408945bf8f4c98346881620a] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=215.9 M 2020-11-17 06:33:18,437 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:33:18,440 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:33:18,463 INFO [Thread-2171] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:33:31,336 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594703473 with entries=32949, filesize=128.33 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594811295 2020-11-17 06:33:45,421 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/0e4e88984b85473097cbad43fa3340c6] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:34:05,003 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#470 average throughput is 54.32 MB/second, slept 47 time(s) and total slept time is 17906 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:34:05,080 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 0e4e88984b85473097cbad43fa3340c6(size=218.9 M), total size for store is 49.2 G. This selection was in queue for 0sec, and took 46sec to execute. 2020-11-17 06:34:05,080 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605594798404; duration=46sec 2020-11-17 06:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=175.24 MB, max=1.83 GB, blockCount=44123, accesses=20795684, hits=17657475, hitRatio=84.91%, , cachingAccesses=17708981, cachingHits=17653057, cachingHitsRatio=99.68%, evictions=2549, evicted=0, evictedPerRun=0.0 2020-11-17 06:34:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 58885 ns. 2020-11-17 06:35:38,704 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 318884 ns. 2020-11-17 06:35:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 107791 ns. 2020-11-17 06:35:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 80032 ns. 2020-11-17 06:35:45,992 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594811295 with entries=40296, filesize=128.40 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594945964 2020-11-17 06:36:39,208 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 51574 ns. 2020-11-17 06:36:47,998 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:37:06,350 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=86.12 MB heapSize=128.00 MB 2020-11-17 06:37:06,387 INFO [Thread-2183] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:37:07,443 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=86.12 MB at sequenceid=261736351 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/ecc3b3935ab64a15af3bb4a34ae2bed2 2020-11-17 06:37:07,451 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:37:07,469 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/ecc3b3935ab64a15af3bb4a34ae2bed2, entries=573394, sequenceid=261736351, filesize=8.9 M 2020-11-17 06:37:07,471 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~86.12 MB/90306127, heapSize ~128.00 MB/134218888, currentSize=43.08 KB/44111 for 73aae5715542e1a5370f7540d2117abc in 1121ms, sequenceid=261736351, compaction requested=true 2020-11-17 06:37:09,560 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 46972 ns. 2020-11-17 06:37:09,560 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 26822 ns. 2020-11-17 06:37:17,416 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.64 MB heapSize=128.05 MB 2020-11-17 06:37:17,443 INFO [Thread-2185] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:37:18,254 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.64 MB at sequenceid=263222318 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/8d54bdfdf8e64746a4a611b57d75c8ad 2020-11-17 06:37:18,259 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:37:18,275 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8d54bdfdf8e64746a4a611b57d75c8ad, entries=366801, sequenceid=263222318, filesize=7.9 M 2020-11-17 06:37:18,278 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.64 MB/106578402, heapSize ~128.05 MB/134266952, currentSize=170.62 KB/174716 for e5dc5c620be19b02666a19c280ff582e in 862ms, sequenceid=263222318, compaction requested=true 2020-11-17 06:37:30,315 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:37:35,957 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594945964 with entries=36557, filesize=128.13 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595055928 2020-11-17 06:38:52,694 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595055928 with entries=25773, filesize=128.72 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595132638 2020-11-17 06:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=175.86 MB, max=1.83 GB, blockCount=44106, accesses=20798157, hits=17659944, hitRatio=84.91%, , cachingAccesses=17711454, cachingHits=17655526, cachingHitsRatio=99.68%, evictions=2579, evicted=0, evictedPerRun=0.0 2020-11-17 06:39:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 35652 ns. 2020-11-17 06:39:57,179 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.98 MB heapSize=128.25 MB 2020-11-17 06:39:57,211 INFO [Thread-2194] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:39:57,917 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.98 MB at sequenceid=272022284 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/1017ef8fdf2c48e4b8e23927fcca85d1 2020-11-17 06:39:57,924 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:39:57,943 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/1017ef8fdf2c48e4b8e23927fcca85d1, entries=319967, sequenceid=272022284, filesize=8.7 M 2020-11-17 06:39:57,944 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.98 MB/110074485, heapSize ~128.25 MB/134475112, currentSize=27.87 KB/28534 for 0469ce862509756add4a8cdb00bc465c in 765ms, sequenceid=272022284, compaction requested=true 2020-11-17 06:40:10,175 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:40:34,203 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.02 MB heapSize=128.14 MB 2020-11-17 06:40:34,231 INFO [Thread-2198] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:40:34,857 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.02 MB at sequenceid=251988035 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/2b125a12721145b185f9ff01dbf83a20 2020-11-17 06:40:34,863 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:40:34,881 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/2b125a12721145b185f9ff01dbf83a20, entries=329367, sequenceid=251988035, filesize=6.8 M 2020-11-17 06:40:34,895 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.02 MB/110116989, heapSize ~128.13 MB/134359280, currentSize=1.05 MB/1104856 for 82421ce8c5c8b48c573baee83caa8305 in 692ms, sequenceid=251988035, compaction requested=true 2020-11-17 06:40:34,896 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 06:40:34,897 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/f0c1d88f56a043cd9c21ac7756575fce, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/1f01dbad25ae420184aa2f30ef838f2f, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/2b125a12721145b185f9ff01dbf83a20] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=96.7 M 2020-11-17 06:40:34,950 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:40:34,954 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:40:34,983 INFO [Thread-2200] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:40:40,235 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595132638 with entries=35160, filesize=128.13 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595240215 2020-11-17 06:40:58,045 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.03 MB heapSize=128.00 MB 2020-11-17 06:40:58,080 INFO [Thread-2203] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:40:58,779 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.03 MB at sequenceid=272979709 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/fa25194bcc994a029dc32a783a18b992 2020-11-17 06:40:58,788 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:40:58,804 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/fa25194bcc994a029dc32a783a18b992, entries=273782, sequenceid=272979709, filesize=8.5 M 2020-11-17 06:40:58,806 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.03 MB/113280753, heapSize ~128.00 MB/134221160, currentSize=991.18 KB/1014969 for fc93f6dcc564341dcc983bf30971f73d in 761ms, sequenceid=272979709, compaction requested=true 2020-11-17 06:40:59,989 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:41:01,790 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:41:01,792 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:41:02,710 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#475 average throughput is 57.04 MB/second, slept 30 time(s) and total slept time is 12744 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:41:02,772 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 472d1003f86b4a9fb43bb927678926f3(size=96.5 M), total size for store is 11.2 G. This selection was in queue for 0sec, and took 27sec to execute. 2020-11-17 06:41:02,772 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605595234896; duration=27sec 2020-11-17 06:41:16,253 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.70 MB heapSize=128.21 MB 2020-11-17 06:41:16,296 INFO [Thread-2205] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:41:17,186 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.70 MB at sequenceid=263233044 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/79882623b9e94df685983145342215b0 2020-11-17 06:41:17,192 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:41:17,217 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/79882623b9e94df685983145342215b0, entries=410622, sequenceid=263233044, filesize=8.1 M 2020-11-17 06:41:17,218 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.70 MB/103496729, heapSize ~128.21 MB/134435640, currentSize=181.55 KB/185911 for e5dc5c620be19b02666a19c280ff582e in 965ms, sequenceid=263233044, compaction requested=true 2020-11-17 06:41:38,991 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 316576 ns. 2020-11-17 06:41:38,991 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 84337 ns. 2020-11-17 06:41:38,991 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 105348 ns. 2020-11-17 06:42:03,120 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595240215 with entries=25513, filesize=128.13 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595323092 2020-11-17 06:42:19,230 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 17079 ns. 2020-11-17 06:42:19,230 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 24761 ns. 2020-11-17 06:42:38,802 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 60501 ns. 2020-11-17 06:43:01,047 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:43:03,309 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.80 MB heapSize=128.00 MB 2020-11-17 06:43:03,346 INFO [Thread-2212] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:43:04,335 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.80 MB at sequenceid=287583460 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/7fede754061847628582af2a3523ea13 2020-11-17 06:43:04,344 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:43:04,364 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/7fede754061847628582af2a3523ea13, entries=519402, sequenceid=287583460, filesize=9.1 M 2020-11-17 06:43:04,365 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.80 MB/95205619, heapSize ~128.00 MB/134218680, currentSize=30.38 KB/31110 for 5e5f05205d7e40b0141cd977dd009ba1 in 1057ms, sequenceid=287583460, compaction requested=true 2020-11-17 06:43:18,059 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:43:41,106 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=98.90 MB heapSize=128.49 MB 2020-11-17 06:43:41,138 INFO [Thread-2215] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:43:41,984 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.90 MB at sequenceid=259913779 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/1acd34fb38be49749de32d8acad381a5 2020-11-17 06:43:41,990 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:43:42,014 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/1acd34fb38be49749de32d8acad381a5, entries=406486, sequenceid=259913779, filesize=7.8 M 2020-11-17 06:43:42,015 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.90 MB/103703251, heapSize ~128.49 MB/134732600, currentSize=217.67 KB/222889 for 06f7ca9534710e37d90d9f8345864259 in 909ms, sequenceid=259913779, compaction requested=true 2020-11-17 06:43:50,688 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595323092 with entries=35357, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595430667 2020-11-17 06:43:50,689 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593823309 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605593823309 2020-11-17 06:43:50,699 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605593912067 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605593912067 2020-11-17 06:43:50,708 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594041731 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605594041731 2020-11-17 06:43:50,715 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594174628 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605594174628 2020-11-17 06:44:33,307 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.27 MB heapSize=128.98 MB 2020-11-17 06:44:33,343 INFO [Thread-2219] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:44:34,080 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.27 MB at sequenceid=263241731 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/17ff1e4a092f467fbf2f34e0b4b9032e 2020-11-17 06:44:34,086 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:44:34,106 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/17ff1e4a092f467fbf2f34e0b4b9032e, entries=356158, sequenceid=263241731, filesize=8.1 M 2020-11-17 06:44:34,114 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.27 MB/108289688, heapSize ~128.98 MB/135240216, currentSize=1.11 MB/1163450 for e5dc5c620be19b02666a19c280ff582e in 807ms, sequenceid=263241731, compaction requested=true 2020-11-17 06:44:34,115 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 06:44:34,115 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8d54bdfdf8e64746a4a611b57d75c8ad, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/79882623b9e94df685983145342215b0, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/17ff1e4a092f467fbf2f34e0b4b9032e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.1 M 2020-11-17 06:44:34,145 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:44:34,147 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:44:34,170 INFO [Thread-2221] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=176.05 MB, max=1.83 GB, blockCount=44102, accesses=20880322, hits=17714569, hitRatio=84.84%, , cachingAccesses=17766078, cachingHits=17710144, cachingHitsRatio=99.69%, evictions=2609, evicted=0, evictedPerRun=0.0 2020-11-17 06:44:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 67580 ns. 2020-11-17 06:44:39,348 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#481 average throughput is 58.45 MB/second, slept 6 time(s) and total slept time is 2047 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 06:44:39,406 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 253d2e0198d0419985492e430e3d4de2(size=25.0 M), total size for store is 49.2 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 06:44:39,406 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605595474114; duration=5sec 2020-11-17 06:44:56,120 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:45:11,298 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595430667 with entries=26957, filesize=128.64 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595511259 2020-11-17 06:45:12,855 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:45:17,067 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:45:21,054 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:46:01,596 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:46:20,486 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:46:42,853 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.31 MB heapSize=128.04 MB 2020-11-17 06:46:42,880 INFO [Thread-2228] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:46:43,546 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.31 MB at sequenceid=251997075 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/b4216ba39a224350b3fafe4f16e9bf83 2020-11-17 06:46:43,552 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:46:43,568 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b4216ba39a224350b3fafe4f16e9bf83, entries=310747, sequenceid=251997075, filesize=6.4 M 2020-11-17 06:46:43,569 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.31 MB/111473182, heapSize ~128.04 MB/134260008, currentSize=332.87 KB/340855 for 82421ce8c5c8b48c573baee83caa8305 in 716ms, sequenceid=251997075, compaction requested=true 2020-11-17 06:47:05,385 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:47:05,939 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595511259 with entries=38941, filesize=128.62 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595625900 2020-11-17 06:47:25,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:47:29,344 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 19377 ns. 2020-11-17 06:47:29,344 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 41592 ns. 2020-11-17 06:47:39,088 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 57717 ns. 2020-11-17 06:47:39,088 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 464276 ns. 2020-11-17 06:47:39,088 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 84526 ns. 2020-11-17 06:47:39,089 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 94612 ns. 2020-11-17 06:48:22,065 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595625900 with entries=26384, filesize=128.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595702020 2020-11-17 06:48:35,216 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.16 MB heapSize=128.00 MB 2020-11-17 06:48:35,267 INFO [Thread-2235] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:48:36,046 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.16 MB at sequenceid=263253040 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/88b3ad865bab4cd1930d2d35c1864c3f 2020-11-17 06:48:36,054 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:48:36,070 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/88b3ad865bab4cd1930d2d35c1864c3f, entries=358436, sequenceid=263253040, filesize=8.1 M 2020-11-17 06:48:36,071 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.16 MB/107123736, heapSize ~128.00 MB/134218152, currentSize=28 KB/28672 for e5dc5c620be19b02666a19c280ff582e in 855ms, sequenceid=263253040, compaction requested=true 2020-11-17 06:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.94 MB, max=1.83 GB, blockCount=44142, accesses=21151764, hits=17982920, hitRatio=85.02%, , cachingAccesses=18034471, cachingHits=17978495, cachingHitsRatio=99.69%, evictions=2639, evicted=0, evictedPerRun=0.0 2020-11-17 06:49:39,376 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 44355 ns. 2020-11-17 06:50:18,900 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595702020 with entries=38798, filesize=128.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595818872 2020-11-17 06:50:39,749 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:51:25,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:51:32,089 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:51:40,594 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595818872 with entries=25950, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595900559 2020-11-17 06:51:47,365 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:51:51,920 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:51:58,949 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:52:04,664 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:52:10,797 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:52:21,466 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:52:29,582 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:52:33,805 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:52:35,612 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.40 MB heapSize=128.15 MB 2020-11-17 06:52:35,646 INFO [Thread-2246] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:52:36,253 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.40 MB at sequenceid=252005768 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/dca1f5c557494cfaab3a721df85122e7 2020-11-17 06:52:36,278 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/dca1f5c557494cfaab3a721df85122e7, entries=310878, sequenceid=252005768, filesize=6.4 M 2020-11-17 06:52:36,280 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.40 MB/111571443, heapSize ~128.15 MB/134370552, currentSize=378.58 KB/387671 for 82421ce8c5c8b48c573baee83caa8305 in 668ms, sequenceid=252005768, compaction requested=true 2020-11-17 06:52:36,282 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 06:52:36,282 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/472d1003f86b4a9fb43bb927678926f3, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b4216ba39a224350b3fafe4f16e9bf83, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/dca1f5c557494cfaab3a721df85122e7] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=109.3 M 2020-11-17 06:52:36,314 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:52:36,326 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:52:36,352 INFO [Thread-2248] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:52:38,074 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:52:38,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 35000 ns. 2020-11-17 06:52:38,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14960 ns. 2020-11-17 06:52:48,418 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:04,221 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:06,914 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:07,679 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#485 average throughput is 57.18 MB/second, slept 34 time(s) and total slept time is 14685 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:53:07,750 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 51f322e8711740869385756e30c8c086(size=109.0 M), total size for store is 11.2 G. This selection was in queue for 0sec, and took 31sec to execute. 2020-11-17 06:53:07,750 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605595956280; duration=31sec 2020-11-17 06:53:10,950 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.57 MB heapSize=128.00 MB 2020-11-17 06:53:10,989 INFO [Thread-2251] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:11,901 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.57 MB at sequenceid=263265143 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/37aa454f5e944a0d88557e7f06356bfa 2020-11-17 06:53:11,907 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:11,939 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/37aa454f5e944a0d88557e7f06356bfa, entries=409707, sequenceid=263265143, filesize=8.1 M 2020-11-17 06:53:11,941 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~98.57 MB/103357654, heapSize ~128.00 MB/134220896, currentSize=51.32 KB/52550 for e5dc5c620be19b02666a19c280ff582e in 991ms, sequenceid=263265143, compaction requested=true 2020-11-17 06:53:11,942 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 06:53:11,942 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/253d2e0198d0419985492e430e3d4de2, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/88b3ad865bab4cd1930d2d35c1864c3f, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/37aa454f5e944a0d88557e7f06356bfa] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.2 M 2020-11-17 06:53:11,997 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:12,001 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:12,031 INFO [Thread-2253] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:12,528 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:16,370 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:21,199 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#487 average throughput is 54.91 MB/second, slept 8 time(s) and total slept time is 2741 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:53:21,279 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 3ebe807feefe44e5932fbfedaabc8db0(size=42.6 M), total size for store is 49.2 G. This selection was in queue for 0sec, and took 9sec to execute. 2020-11-17 06:53:21,279 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605595991942; duration=9sec 2020-11-17 06:53:21,824 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.49 MB heapSize=128.53 MB 2020-11-17 06:53:21,854 INFO [Thread-2255] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:22,639 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.49 MB at sequenceid=272057630 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/c3ab2c9432b743f7b95af296c6530dd1 2020-11-17 06:53:22,662 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/c3ab2c9432b743f7b95af296c6530dd1, entries=316670, sequenceid=272057630, filesize=8.7 M 2020-11-17 06:53:22,665 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.49 MB/110609779, heapSize ~128.53 MB/134769712, currentSize=34.00 KB/34819 for 0469ce862509756add4a8cdb00bc465c in 840ms, sequenceid=272057630, compaction requested=true 2020-11-17 06:53:22,666 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 06:53:22,666 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/e5cef750c77e4b61a081a737d4e259cf, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/1017ef8fdf2c48e4b8e23927fcca85d1, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/c3ab2c9432b743f7b95af296c6530dd1] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=62.5 M 2020-11-17 06:53:22,718 INFO [Thread-2257] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:22,751 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:26,867 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:31,488 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595900559 with entries=34043, filesize=128.37 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596011466 2020-11-17 06:53:35,136 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#489 average throughput is 58.90 MB/second, slept 13 time(s) and total slept time is 5383 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:53:35,208 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into b8829e5919ba4868808fe5dcbee0ca43(size=63.3 M), total size for store is 42.8 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 06:53:35,208 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605596002665; duration=12sec 2020-11-17 06:53:39,389 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 49293 ns. 2020-11-17 06:53:39,389 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 113338 ns. 2020-11-17 06:53:39,389 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 251372 ns. 2020-11-17 06:53:39,389 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 81185 ns. 2020-11-17 06:53:44,525 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:49,414 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:50,529 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.17 MB heapSize=128.05 MB 2020-11-17 06:53:50,562 INFO [Thread-2260] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:51,639 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.17 MB at sequenceid=261783385 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/2c6983c3869741ea8acad106d5395446 2020-11-17 06:53:51,668 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/2c6983c3869741ea8acad106d5395446, entries=586273, sequenceid=261783385, filesize=8.9 M 2020-11-17 06:53:51,675 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.17 MB/89303955, heapSize ~128.05 MB/134265488, currentSize=354.25 KB/362753 for 73aae5715542e1a5370f7540d2117abc in 1146ms, sequenceid=261783385, compaction requested=true 2020-11-17 06:53:51,676 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 06:53:51,676 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/5df2c588d5014764a66a5922767e70cf, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/ecc3b3935ab64a15af3bb4a34ae2bed2, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/2c6983c3869741ea8acad106d5395446] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=26.3 M 2020-11-17 06:53:51,706 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:51,731 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:51,760 INFO [Thread-2262] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:52,610 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:53:56,117 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#491 average throughput is 58.05 MB/second, slept 5 time(s) and total slept time is 858 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:53:56,209 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 8c5098f251674474bcb633bdc50d5d40(size=26.9 M), total size for store is 12.6 G. This selection was in queue for 0sec, and took 4sec to execute. 2020-11-17 06:53:56,209 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=10, startTime=1605596031675; duration=4sec 2020-11-17 06:54:02,012 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:54:09,862 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.43 MB, max=1.83 GB, blockCount=44196, accesses=21913763, hits=18691572, hitRatio=85.30%, , cachingAccesses=18743182, cachingHits=18687088, cachingHitsRatio=99.70%, evictions=2669, evicted=0, evictedPerRun=0.0 2020-11-17 06:54:43,396 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596011466 with entries=21642, filesize=129.20 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596083343 2020-11-17 06:55:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 49622 ns. 2020-11-17 06:55:43,756 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.59 MB heapSize=128.00 MB 2020-11-17 06:55:43,787 INFO [Thread-2268] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:55:44,651 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.59 MB at sequenceid=259947406 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/879b9f81fc93424d898f75cb0ff606c4 2020-11-17 06:55:44,657 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:55:44,672 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/879b9f81fc93424d898f75cb0ff606c4, entries=380889, sequenceid=259947406, filesize=7.2 M 2020-11-17 06:55:44,674 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.59 MB/105475310, heapSize ~128.00 MB/134218216, currentSize=201.22 KB/206048 for 06f7ca9534710e37d90d9f8345864259 in 918ms, sequenceid=259947406, compaction requested=true 2020-11-17 06:55:44,675 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 06:55:44,675 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/70b061d3fd0d4ef5b609e85df0fba5ae, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/1acd34fb38be49749de32d8acad381a5, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/879b9f81fc93424d898f75cb0ff606c4] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=37.1 M 2020-11-17 06:55:44,703 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:55:44,706 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:55:44,729 INFO [Thread-2270] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:55:53,582 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#493 average throughput is 55.98 MB/second, slept 8 time(s) and total slept time is 3631 ms. 0 active operations remaining, total limit is 53.85 MB/second 2020-11-17 06:55:53,638 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 41bd92b6629c418cb0dc506e95dfd9c3(size=37.5 M), total size for store is 55.4 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 06:55:53,638 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=11, startTime=1605596144674; duration=8sec 2020-11-17 06:56:11,131 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596083343 with entries=29457, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596171094 2020-11-17 06:56:21,941 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.51 MB heapSize=128.38 MB 2020-11-17 06:56:21,979 INFO [Thread-2274] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:56:22,879 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.51 MB at sequenceid=263273642 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/ccf8d8dab4854cdba9d2678a7f502d21 2020-11-17 06:56:22,896 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:56:22,918 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ccf8d8dab4854cdba9d2678a7f502d21, entries=386874, sequenceid=263273642, filesize=7.7 M 2020-11-17 06:56:22,919 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.51 MB/105392901, heapSize ~128.38 MB/134613168, currentSize=532.44 KB/545219 for e5dc5c620be19b02666a19c280ff582e in 979ms, sequenceid=263273642, compaction requested=true 2020-11-17 06:56:28,700 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.13 MB heapSize=128.40 MB 2020-11-17 06:56:28,722 INFO [Thread-2276] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:56:29,432 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.13 MB at sequenceid=273002245 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/d43b4c9a672c4101b1745c5529760fef 2020-11-17 06:56:29,438 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:56:29,456 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/d43b4c9a672c4101b1745c5529760fef, entries=278128, sequenceid=273002245, filesize=8.5 M 2020-11-17 06:56:29,458 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.13 MB/113381784, heapSize ~128.40 MB/134634512, currentSize=53.43 KB/54708 for fc93f6dcc564341dcc983bf30971f73d in 757ms, sequenceid=273002245, compaction requested=true 2020-11-17 06:56:29,458 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 06:56:29,458 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/cf40bb813d3f48b1921ee1644a017db9, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/fa25194bcc994a029dc32a783a18b992, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/d43b4c9a672c4101b1745c5529760fef] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=60.7 M 2020-11-17 06:56:29,486 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:56:29,490 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:56:29,510 INFO [Thread-2278] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:56:37,437 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:56:43,217 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#496 average throughput is 55.08 MB/second, slept 15 time(s) and total slept time is 7343 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 06:56:43,282 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into 4cd358d4b94b45a080ce413303beeea1(size=61.4 M), total size for store is 33.4 G. This selection was in queue for 0sec, and took 13sec to execute. 2020-11-17 06:56:43,282 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=11, startTime=1605596189458; duration=13sec 2020-11-17 06:56:47,999 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:57:00,280 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:57:39,293 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14429 ns. 2020-11-17 06:57:39,293 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 43373 ns. 2020-11-17 06:57:44,036 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596171094 with entries=30010, filesize=128.40 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596264007 2020-11-17 06:57:58,302 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:58:08,782 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.52 MB heapSize=128.30 MB 2020-11-17 06:58:08,814 INFO [Thread-2286] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:58:09,448 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.52 MB at sequenceid=252013537 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/d63aaa6c1dd74f31b01822d5075ea9ff 2020-11-17 06:58:09,454 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:58:09,471 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d63aaa6c1dd74f31b01822d5075ea9ff, entries=311404, sequenceid=252013537, filesize=6.4 M 2020-11-17 06:58:09,474 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.52 MB/111695933, heapSize ~128.30 MB/134535856, currentSize=475.77 KB/487185 for 82421ce8c5c8b48c573baee83caa8305 in 692ms, sequenceid=252013537, compaction requested=true 2020-11-17 06:58:32,797 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:58:39,652 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 51748 ns. 2020-11-17 06:59:22,195 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.63 MB heapSize=128.00 MB 2020-11-17 06:59:22,242 INFO [Thread-2291] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:59:23,495 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.63 MB at sequenceid=291493020 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/9590d29e15e140669096ddbcb3d1b79a 2020-11-17 06:59:23,500 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 06:59:23,517 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/9590d29e15e140669096ddbcb3d1b79a, entries=547221, sequenceid=291493020, filesize=8.9 M 2020-11-17 06:59:23,519 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~88.63 MB/92939004, heapSize ~128.00 MB/134218296, currentSize=42.34 KB/43359 for bdaa31a63b7f3d06d9909b7560333e79 in 1324ms, sequenceid=291493020, compaction requested=true 2020-11-17 06:59:30,491 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596264007 with entries=35143, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596370470 2020-11-17 06:59:30,492 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594296003 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605594296003 2020-11-17 06:59:30,499 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594418260 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605594418260 2020-11-17 06:59:30,505 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594579707 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605594579707 2020-11-17 06:59:30,510 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594703473 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605594703473 2020-11-17 06:59:30,516 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594811295 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605594811295 2020-11-17 06:59:30,521 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605594945964 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605594945964 2020-11-17 06:59:30,527 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595055928 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605595055928 2020-11-17 06:59:30,532 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595132638 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605595132638 2020-11-17 06:59:30,536 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595240215 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605595240215 2020-11-17 06:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.43 MB, max=1.83 GB, blockCount=44165, accesses=21950861, hits=18708383, hitRatio=85.23%, , cachingAccesses=18759999, cachingHits=18703881, cachingHitsRatio=99.70%, evictions=2699, evicted=0, evictedPerRun=0.0 2020-11-17 06:59:38,800 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 255801 ns. 2020-11-17 06:59:38,800 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 82242 ns. 2020-11-17 06:59:38,800 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 120057 ns. 2020-11-17 07:00:09,040 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.85 MB heapSize=128.16 MB 2020-11-17 07:00:09,091 INFO [Thread-2294] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:00:09,819 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.85 MB at sequenceid=263283571 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/e9c88b74df9748ae87f54c9332e0fc3e 2020-11-17 07:00:09,826 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:00:09,848 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e9c88b74df9748ae87f54c9332e0fc3e, entries=364947, sequenceid=263283571, filesize=8.1 M 2020-11-17 07:00:09,849 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.85 MB/106795542, heapSize ~128.16 MB/134380992, currentSize=1.35 MB/1418642 for e5dc5c620be19b02666a19c280ff582e in 809ms, sequenceid=263283571, compaction requested=true 2020-11-17 07:00:09,850 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 07:00:09,850 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3ebe807feefe44e5932fbfedaabc8db0, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ccf8d8dab4854cdba9d2678a7f502d21, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e9c88b74df9748ae87f54c9332e0fc3e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=58.5 M 2020-11-17 07:00:09,876 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:00:09,878 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:00:09,900 INFO [Thread-2296] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:00:21,135 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:00:22,152 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#500 average throughput is 57.60 MB/second, slept 14 time(s) and total slept time is 4903 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 07:00:22,225 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 4e491e17404c44b787f7dc5ab9c21055(size=60.0 M), total size for store is 49.2 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 07:00:22,225 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605596409849; duration=12sec 2020-11-17 07:00:33,260 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:00:39,204 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 31432 ns. 2020-11-17 07:00:39,400 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:00:47,444 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596370470 with entries=24565, filesize=128.46 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596447403 2020-11-17 07:00:49,195 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:00:58,435 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:02:02,612 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:02:06,952 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:02:30,547 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596447403 with entries=35289, filesize=128.10 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596550508 2020-11-17 07:02:49,021 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 40322 ns. 2020-11-17 07:02:49,021 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 10240 ns. 2020-11-17 07:03:21,818 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:03:41,670 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:03:54,629 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.77 MB heapSize=128.01 MB 2020-11-17 07:03:54,721 INFO [Thread-2308] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:03:55,761 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.77 MB at sequenceid=263294006 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/841a03d238e74b59b1a9f7df51353447 2020-11-17 07:03:55,777 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:03:55,796 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/841a03d238e74b59b1a9f7df51353447, entries=349789, sequenceid=263294006, filesize=8.0 M 2020-11-17 07:03:55,797 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.77 MB/107766402, heapSize ~128.01 MB/134223536, currentSize=34.77 KB/35608 for e5dc5c620be19b02666a19c280ff582e in 1168ms, sequenceid=263294006, compaction requested=true 2020-11-17 07:04:02,123 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:04:11,130 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596550508 with entries=33555, filesize=128.08 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596651094 2020-11-17 07:04:22,973 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.32 MB heapSize=128.23 MB 2020-11-17 07:04:23,012 INFO [Thread-2312] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:04:23,678 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.32 MB at sequenceid=252022767 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/8eaba7086d0e40a1b2ef39664da010a1 2020-11-17 07:04:23,685 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:04:23,711 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/8eaba7086d0e40a1b2ef39664da010a1, entries=313196, sequenceid=252022767, filesize=6.4 M 2020-11-17 07:04:23,713 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.32 MB/111481302, heapSize ~128.23 MB/134462320, currentSize=451.97 KB/462813 for 82421ce8c5c8b48c573baee83caa8305 in 740ms, sequenceid=252022767, compaction requested=true 2020-11-17 07:04:23,714 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 07:04:23,714 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/51f322e8711740869385756e30c8c086, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d63aaa6c1dd74f31b01822d5075ea9ff, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/8eaba7086d0e40a1b2ef39664da010a1] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=121.8 M 2020-11-17 07:04:23,750 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:04:23,753 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:04:23,778 INFO [Thread-2314] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.29 MB, max=1.83 GB, blockCount=44173, accesses=22234889, hits=18969216, hitRatio=85.31%, , cachingAccesses=19020842, cachingHits=18964707, cachingHitsRatio=99.70%, evictions=2729, evicted=0, evictedPerRun=0.0 2020-11-17 07:04:38,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 56895 ns. 2020-11-17 07:04:55,040 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:04:55,042 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:04:57,648 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:04:58,429 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#503 average throughput is 57.73 MB/second, slept 40 time(s) and total slept time is 17284 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 07:04:58,494 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:04:58,517 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 125dc01d08574882823d5bdf8719b7c1(size=121.4 M), total size for store is 11.2 G. This selection was in queue for 0sec, and took 34sec to execute. 2020-11-17 07:04:58,517 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605596663713; duration=34sec 2020-11-17 07:05:09,482 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:05:38,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 150887 ns. 2020-11-17 07:05:38,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 304774 ns. 2020-11-17 07:05:38,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 90827 ns. 2020-11-17 07:05:45,137 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596651094 with entries=32974, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596745116 2020-11-17 07:06:38,757 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 45269 ns. 2020-11-17 07:06:55,787 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.42 MB heapSize=128.11 MB 2020-11-17 07:06:55,845 INFO [Thread-2324] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:06:56,905 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.42 MB at sequenceid=287653920 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/5bfaf5950d5f41b68a2315ad080202d5 2020-11-17 07:06:56,912 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:06:56,931 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/5bfaf5950d5f41b68a2315ad080202d5, entries=526644, sequenceid=287653920, filesize=9.1 M 2020-11-17 07:06:56,933 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.42 MB/94808378, heapSize ~128.11 MB/134335976, currentSize=293.59 KB/300633 for 5e5f05205d7e40b0141cd977dd009ba1 in 1145ms, sequenceid=287653920, compaction requested=true 2020-11-17 07:06:56,934 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 07:06:56,934 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/9c752d45038d4f05ac947dccd82e4630, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/7fede754061847628582af2a3523ea13, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/5bfaf5950d5f41b68a2315ad080202d5] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=120.8 M 2020-11-17 07:06:56,966 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:06:56,973 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:06:56,997 INFO [Thread-2326] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:03,369 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:03,371 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:04,374 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.79 MB heapSize=128.01 MB 2020-11-17 07:07:04,423 INFO [Thread-2328] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:05,121 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.79 MB at sequenceid=272093951 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/764bc398640d4b0892a4f70f2bf3dace 2020-11-17 07:07:05,129 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:05,146 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/764bc398640d4b0892a4f70f2bf3dace, entries=319320, sequenceid=272093951, filesize=8.6 M 2020-11-17 07:07:05,147 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.79 MB/109875863, heapSize ~128.01 MB/134223888, currentSize=36.33 KB/37198 for 0469ce862509756add4a8cdb00bc465c in 773ms, sequenceid=272093951, compaction requested=true 2020-11-17 07:07:12,326 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:12,327 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:12,798 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:12,800 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:14,350 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:15,035 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:15,450 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:15,452 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:17,032 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#505 average throughput is 56.41 MB/second, slept 18 time(s) and total slept time is 5289 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 07:07:17,083 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:17,114 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into ad18bc9861a24f818af5160b588ec702(size=121.4 M), total size for store is 24.3 G. This selection was in queue for 0sec, and took 20sec to execute. 2020-11-17 07:07:17,114 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=11, startTime=1605596816933; duration=20sec 2020-11-17 07:07:20,988 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596745116 with entries=32229, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596840954 2020-11-17 07:07:20,989 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595323092 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605595323092 2020-11-17 07:07:21,001 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595430667 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605595430667 2020-11-17 07:07:21,006 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595511259 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605595511259 2020-11-17 07:07:21,023 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595625900 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605595625900 2020-11-17 07:07:21,030 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595702020 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605595702020 2020-11-17 07:07:21,035 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595818872 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605595818872 2020-11-17 07:07:21,040 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605595900559 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605595900559 2020-11-17 07:07:43,380 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.48 MB heapSize=128.00 MB 2020-11-17 07:07:43,408 INFO [Thread-2331] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:44,175 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.48 MB at sequenceid=259981832 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/e75bb14a8e0e447d8cbf1e4aa978ff57 2020-11-17 07:07:44,180 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:07:44,201 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/e75bb14a8e0e447d8cbf1e4aa978ff57, entries=378684, sequenceid=259981832, filesize=7.2 M 2020-11-17 07:07:44,202 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.48 MB/105358260, heapSize ~128.00 MB/134217744, currentSize=22.79 KB/23340 for 06f7ca9534710e37d90d9f8345864259 in 822ms, sequenceid=259981832, compaction requested=true 2020-11-17 07:07:58,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14536 ns. 2020-11-17 07:07:58,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 30093 ns. 2020-11-17 07:08:10,427 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=97.62 MB heapSize=128.00 MB 2020-11-17 07:08:10,460 INFO [Thread-2334] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:08:11,327 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.62 MB at sequenceid=263305835 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/4e2c8a145a91405392768e04e9ee8e04 2020-11-17 07:08:11,332 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:08:11,350 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/4e2c8a145a91405392768e04e9ee8e04, entries=422858, sequenceid=263305835, filesize=7.9 M 2020-11-17 07:08:11,351 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~97.62 MB/102367131, heapSize ~128.00 MB/134217744, currentSize=357.03 KB/365597 for e5dc5c620be19b02666a19c280ff582e in 924ms, sequenceid=263305835, compaction requested=true 2020-11-17 07:08:11,352 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 07:08:11,352 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/4e491e17404c44b787f7dc5ab9c21055, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/841a03d238e74b59b1a9f7df51353447, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/4e2c8a145a91405392768e04e9ee8e04] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=75.9 M 2020-11-17 07:08:11,380 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:08:11,382 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:08:11,406 INFO [Thread-2336] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:08:18,520 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:08:27,206 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#509 average throughput is 57.34 MB/second, slept 17 time(s) and total slept time is 6723 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 07:08:27,268 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:08:27,290 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 8e25aa8d37374e1dbdbc905a6215d790(size=77.5 M), total size for store is 49.3 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 07:08:27,291 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605596891351; duration=15sec 2020-11-17 07:08:53,018 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596840954 with entries=30203, filesize=128.36 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596932985 2020-11-17 07:08:56,193 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.57 MB heapSize=128.03 MB 2020-11-17 07:08:56,222 INFO [Thread-2341] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:08:57,305 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.57 MB at sequenceid=261826292 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/13e5a13f641a4848b07467ead5ef8a67 2020-11-17 07:08:57,310 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:08:57,327 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/13e5a13f641a4848b07467ead5ef8a67, entries=580855, sequenceid=261826292, filesize=8.8 M 2020-11-17 07:08:57,328 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.57 MB/89730103, heapSize ~128.03 MB/134248648, currentSize=139.92 KB/143280 for 73aae5715542e1a5370f7540d2117abc in 1135ms, sequenceid=261826292, compaction requested=true 2020-11-17 07:09:09,802 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.87 MB, max=1.83 GB, blockCount=44158, accesses=22291931, hits=18972229, hitRatio=85.11%, , cachingAccesses=19023847, cachingHits=18967698, cachingHitsRatio=99.70%, evictions=2759, evicted=0, evictedPerRun=0.0 2020-11-17 07:10:39,453 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 1107481 ns. 2020-11-17 07:10:39,897 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596932985 with entries=34538, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597039865 2020-11-17 07:10:39,898 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596011466 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605596011466 2020-11-17 07:10:39,908 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596083343 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605596083343 2020-11-17 07:10:43,478 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.07 MB heapSize=128.05 MB 2020-11-17 07:10:43,500 INFO [Thread-2347] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:10:44,074 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.07 MB at sequenceid=252032312 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/d6c4eca040e547c683766e66f208545f 2020-11-17 07:10:44,080 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:10:44,098 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d6c4eca040e547c683766e66f208545f, entries=314053, sequenceid=252032312, filesize=6.4 M 2020-11-17 07:10:44,099 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.07 MB/111221389, heapSize ~128.05 MB/134271840, currentSize=891.13 KB/912518 for 82421ce8c5c8b48c573baee83caa8305 in 621ms, sequenceid=252032312, compaction requested=true 2020-11-17 07:11:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 272457 ns. 2020-11-17 07:11:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 84286 ns. 2020-11-17 07:11:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 90176 ns. 2020-11-17 07:11:43,552 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.13 MB heapSize=128.11 MB 2020-11-17 07:11:43,584 INFO [Thread-2352] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:11:44,339 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.13 MB at sequenceid=263315549 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a6c34b2405d144a18decc5c5356660a3 2020-11-17 07:11:44,347 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:11:44,363 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a6c34b2405d144a18decc5c5356660a3, entries=374447, sequenceid=263315549, filesize=8.2 M 2020-11-17 07:11:44,365 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.13 MB/106041512, heapSize ~128.11 MB/134330968, currentSize=139.84 KB/143196 for e5dc5c620be19b02666a19c280ff582e in 813ms, sequenceid=263315549, compaction requested=true 2020-11-17 07:11:47,658 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.92 MB heapSize=128.12 MB 2020-11-17 07:11:47,687 INFO [Thread-2354] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:11:48,339 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.92 MB at sequenceid=273025327 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/ba1b068ce8914983a93d2c23419dd9ff 2020-11-17 07:11:48,390 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:11:48,430 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/ba1b068ce8914983a93d2c23419dd9ff, entries=277127, sequenceid=273025327, filesize=8.5 M 2020-11-17 07:11:48,433 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~107.92 MB/113161992, heapSize ~128.12 MB/134339080, currentSize=648.00 KB/663555 for fc93f6dcc564341dcc983bf30971f73d in 776ms, sequenceid=273025327, compaction requested=true 2020-11-17 07:12:02,559 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597039865 with entries=28263, filesize=128.26 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597122531 2020-11-17 07:12:02,559 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596171094 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605596171094 2020-11-17 07:12:27,981 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:12:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 52443 ns. 2020-11-17 07:12:58,763 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 21310 ns. 2020-11-17 07:12:58,763 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 26477 ns. 2020-11-17 07:13:00,498 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:13:18,072 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:13:31,876 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:13:41,348 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:13:48,236 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597122531 with entries=35255, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597228177 2020-11-17 07:13:59,388 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:14:12,021 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:14:16,200 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.43 MB, max=1.83 GB, blockCount=44173, accesses=22301836, hits=18982109, hitRatio=85.11%, , cachingAccesses=19033752, cachingHits=18977578, cachingHitsRatio=99.70%, evictions=2789, evicted=0, evictedPerRun=0.0 2020-11-17 07:14:57,385 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.43 MB heapSize=128.23 MB 2020-11-17 07:14:57,437 INFO [Thread-2363] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:14:58,237 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.43 MB at sequenceid=263324580 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/7e5ac4dc5a1745169a2d934a72779050 2020-11-17 07:14:58,244 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:14:58,274 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7e5ac4dc5a1745169a2d934a72779050, entries=343496, sequenceid=263324580, filesize=8.1 M 2020-11-17 07:14:58,275 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.43 MB/108457394, heapSize ~128.23 MB/134462336, currentSize=558.00 KB/571396 for e5dc5c620be19b02666a19c280ff582e in 890ms, sequenceid=263324580, compaction requested=true 2020-11-17 07:14:58,277 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 07:14:58,277 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8e25aa8d37374e1dbdbc905a6215d790, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a6c34b2405d144a18decc5c5356660a3, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7e5ac4dc5a1745169a2d934a72779050] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=93.8 M 2020-11-17 07:14:58,318 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:14:58,321 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:14:58,356 INFO [Thread-2365] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:15:00,234 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597228177 with entries=25641, filesize=128.41 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597300214 2020-11-17 07:15:10,441 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:15:18,261 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#515 average throughput is 55.78 MB/second, slept 20 time(s) and total slept time is 7919 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 07:15:18,366 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 0c540fc1fa6240ab8958280c45720d11(size=95.4 M), total size for store is 49.3 G. This selection was in queue for 0sec, and took 20sec to execute. 2020-11-17 07:15:18,366 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605597298276; duration=20sec 2020-11-17 07:15:39,487 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 74085 ns. 2020-11-17 07:15:56,423 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:16:03,544 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:16:10,254 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:16:14,485 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:16:48,289 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.12 MB heapSize=128.03 MB 2020-11-17 07:16:48,330 INFO [Thread-2372] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:16:49,007 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.12 MB at sequenceid=252041685 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/751d8dc7da6b46f8b24c53f9798dfe6b 2020-11-17 07:16:49,014 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:16:49,035 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/751d8dc7da6b46f8b24c53f9798dfe6b, entries=313096, sequenceid=252041685, filesize=6.4 M 2020-11-17 07:16:49,048 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.12 MB/111273721, heapSize ~128.03 MB/134246672, currentSize=487.61 KB/499317 for 82421ce8c5c8b48c573baee83caa8305 in 758ms, sequenceid=252041685, compaction requested=true 2020-11-17 07:17:00,769 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597300214 with entries=40213, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597420713 2020-11-17 07:17:38,755 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 122607 ns. 2020-11-17 07:17:38,755 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 49043 ns. 2020-11-17 07:17:38,755 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 262742 ns. 2020-11-17 07:17:38,755 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 62389 ns. 2020-11-17 07:17:53,267 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:18:01,321 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:18:08,960 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 41913 ns. 2020-11-17 07:18:08,960 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 12450 ns. 2020-11-17 07:18:09,716 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597420713 with entries=23549, filesize=128.15 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597489632 2020-11-17 07:18:54,949 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:19:26,080 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.69 MB heapSize=128.01 MB 2020-11-17 07:19:26,120 INFO [Thread-2380] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:19:26,851 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.69 MB at sequenceid=260015913 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/6d40f1f2399c4cbeb6b1b5fabc148d3a 2020-11-17 07:19:26,860 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:19:26,877 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/6d40f1f2399c4cbeb6b1b5fabc148d3a, entries=377974, sequenceid=260015913, filesize=7.2 M 2020-11-17 07:19:26,879 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.69 MB/105579199, heapSize ~128.01 MB/134223424, currentSize=50.12 KB/51324 for 06f7ca9534710e37d90d9f8345864259 in 800ms, sequenceid=260015913, compaction requested=true 2020-11-17 07:19:26,880 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 07:19:26,880 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/41bd92b6629c418cb0dc506e95dfd9c3, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/e75bb14a8e0e447d8cbf1e4aa978ff57, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/6d40f1f2399c4cbeb6b1b5fabc148d3a] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=51.8 M 2020-11-17 07:19:26,912 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:19:26,918 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:19:26,942 INFO [Thread-2382] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.90 MB, max=1.83 GB, blockCount=44187, accesses=22597912, hits=19251160, hitRatio=85.19%, , cachingAccesses=19302813, cachingHits=19246623, cachingHitsRatio=99.71%, evictions=2819, evicted=0, evictedPerRun=0.0 2020-11-17 07:19:38,649 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#518 average throughput is 59.38 MB/second, slept 13 time(s) and total slept time is 4769 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 07:19:38,774 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 1047b8dd542043fa8baef1b6d8220422(size=52.1 M), total size for store is 55.4 G. This selection was in queue for 0sec, and took 11sec to execute. 2020-11-17 07:19:38,774 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=11, startTime=1605597566879; duration=11sec 2020-11-17 07:19:53,024 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.83 MB heapSize=128.05 MB 2020-11-17 07:19:53,057 INFO [Thread-2386] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:19:53,776 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.83 MB at sequenceid=263338363 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/e42417ffa9804b3e92f0fc86cb146c20 2020-11-17 07:19:53,781 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:19:53,795 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e42417ffa9804b3e92f0fc86cb146c20, entries=392402, sequenceid=263338363, filesize=8.1 M 2020-11-17 07:19:53,797 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~99.83 MB/104679403, heapSize ~128.05 MB/134269192, currentSize=138.90 KB/142238 for e5dc5c620be19b02666a19c280ff582e in 773ms, sequenceid=263338363, compaction requested=true 2020-11-17 07:20:02,293 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597489632 with entries=39428, filesize=128.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597602270 2020-11-17 07:20:38,177 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.87 MB heapSize=128.00 MB 2020-11-17 07:20:38,232 INFO [Thread-2389] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:20:38,931 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.87 MB at sequenceid=272130646 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/3a524c3b3427441cb9eba7711421c0df 2020-11-17 07:20:38,938 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:20:38,954 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/3a524c3b3427441cb9eba7711421c0df, entries=317956, sequenceid=272130646, filesize=8.7 M 2020-11-17 07:20:38,955 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.87 MB/109967020, heapSize ~128.00 MB/134218528, currentSize=19.10 KB/19557 for 0469ce862509756add4a8cdb00bc465c in 778ms, sequenceid=272130646, compaction requested=true 2020-11-17 07:20:38,956 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 07:20:38,956 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/b8829e5919ba4868808fe5dcbee0ca43, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/764bc398640d4b0892a4f70f2bf3dace, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/3a524c3b3427441cb9eba7711421c0df] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=80.6 M 2020-11-17 07:20:38,985 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:20:38,990 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:20:39,009 INFO [Thread-2391] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:20:42,280 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:20:48,211 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:20:53,475 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:20:53,477 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:20:55,129 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#521 average throughput is 58.11 MB/second, slept 17 time(s) and total slept time is 7193 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 07:20:55,184 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into e417228cc7ef4eb2b6509ac64d977982(size=81.2 M), total size for store is 42.8 G. This selection was in queue for 0sec, and took 16sec to execute. 2020-11-17 07:20:55,185 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605597638956; duration=16sec 2020-11-17 07:21:18,382 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:21:18,808 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597602270 with entries=26630, filesize=128.32 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597678780 2020-11-17 07:21:38,658 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 64255 ns. 2020-11-17 07:22:16,844 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.44 MB heapSize=128.28 MB 2020-11-17 07:22:16,887 INFO [Thread-2398] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:22:17,497 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.44 MB at sequenceid=252050455 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/28835c0b823f4857bb51181a6b18a077 2020-11-17 07:22:17,503 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:22:17,523 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/28835c0b823f4857bb51181a6b18a077, entries=312133, sequenceid=252050455, filesize=6.4 M 2020-11-17 07:22:17,524 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.44 MB/111607753, heapSize ~128.28 MB/134508304, currentSize=58.53 KB/59939 for 82421ce8c5c8b48c573baee83caa8305 in 680ms, sequenceid=252050455, compaction requested=true 2020-11-17 07:22:17,525 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 07:22:17,525 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d6c4eca040e547c683766e66f208545f, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/751d8dc7da6b46f8b24c53f9798dfe6b, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/28835c0b823f4857bb51181a6b18a077] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=19.2 M 2020-11-17 07:22:17,561 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:22:17,564 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:22:17,588 INFO [Thread-2400] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:22:23,058 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#523 average throughput is 58.07 MB/second, slept 6 time(s) and total slept time is 2384 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 07:22:23,136 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 549dbe1c90b343d4b9315c031a84bc0d(size=19.1 M), total size for store is 11.2 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 07:22:23,136 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605597737524; duration=5sec 2020-11-17 07:22:38,610 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:22:39,140 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 42201 ns. 2020-11-17 07:22:50,784 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:23:07,398 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597678780 with entries=36541, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597787344 2020-11-17 07:23:12,716 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:23:18,073 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:23:19,234 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 32466 ns. 2020-11-17 07:23:19,234 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 8704 ns. 2020-11-17 07:23:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 110128 ns. 2020-11-17 07:23:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 259962 ns. 2020-11-17 07:23:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 86194 ns. 2020-11-17 07:23:42,893 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.84 MB heapSize=128.14 MB 2020-11-17 07:23:42,924 INFO [Thread-2405] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:23:43,755 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.84 MB at sequenceid=263349120 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/b8a5b654f59140a4b8074f7c3a8c5867 2020-11-17 07:23:43,766 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:23:43,783 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b8a5b654f59140a4b8074f7c3a8c5867, entries=392974, sequenceid=263349120, filesize=8.2 M 2020-11-17 07:23:43,785 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~99.84 MB/104690870, heapSize ~128.14 MB/134362432, currentSize=506.13 KB/518273 for e5dc5c620be19b02666a19c280ff582e in 892ms, sequenceid=263349120, compaction requested=true 2020-11-17 07:23:43,786 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 07:23:43,786 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/0c540fc1fa6240ab8958280c45720d11, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e42417ffa9804b3e92f0fc86cb146c20, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b8a5b654f59140a4b8074f7c3a8c5867] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=111.6 M 2020-11-17 07:23:43,812 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:23:43,814 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:23:43,836 INFO [Thread-2407] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:23:46,198 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:23:57,120 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:24:06,541 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#525 average throughput is 57.72 MB/second, slept 25 time(s) and total slept time is 9047 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 07:24:06,634 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into ebefc79f6f4d4df0aa1bbad6d5362973(size=113.2 M), total size for store is 49.3 G. This selection was in queue for 0sec, and took 22sec to execute. 2020-11-17 07:24:06,635 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605597823785; duration=22sec 2020-11-17 07:24:08,218 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.68 MB heapSize=128.09 MB 2020-11-17 07:24:08,261 INFO [Thread-2410] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:24:09,429 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.68 MB at sequenceid=261870383 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/f500165e564048be82efc29c5c43d15e 2020-11-17 07:24:09,449 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/f500165e564048be82efc29c5c43d15e, entries=575238, sequenceid=261870383, filesize=8.9 M 2020-11-17 07:24:09,450 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.68 MB/89842899, heapSize ~128.09 MB/134311312, currentSize=210.67 KB/215724 for 73aae5715542e1a5370f7540d2117abc in 1232ms, sequenceid=261870383, compaction requested=true 2020-11-17 07:24:09,450 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 07:24:09,450 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/8c5098f251674474bcb633bdc50d5d40, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/13e5a13f641a4848b07467ead5ef8a67, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/f500165e564048be82efc29c5c43d15e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=44.6 M 2020-11-17 07:24:09,478 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:24:09,482 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:24:09,508 INFO [Thread-2412] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:24:12,723 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:24:16,009 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:24:16,778 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#527 average throughput is 57.69 MB/second, slept 7 time(s) and total slept time is 1344 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 07:24:16,844 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 21f7ec74002149bd8f49b23fea1169b2(size=45.2 M), total size for store is 12.6 G. This selection was in queue for 0sec, and took 7sec to execute. 2020-11-17 07:24:16,844 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=10, startTime=1605597849450; duration=7sec 2020-11-17 07:24:19,981 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:24:22,843 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597787344 with entries=26018, filesize=128.65 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597862820 2020-11-17 07:24:25,150 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:24:32,429 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.24 MB, max=1.83 GB, blockCount=44181, accesses=22663901, hits=19266201, hitRatio=85.01%, , cachingAccesses=19317841, cachingHits=19261620, cachingHitsRatio=99.71%, evictions=2849, evicted=0, evictedPerRun=0.0 2020-11-17 07:24:40,989 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:24:52,614 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:25:10,517 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:25:19,462 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:25:22,594 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:25:25,861 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:25:33,510 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:25:40,815 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:25:59,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:26:04,576 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:26:11,631 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597862820 with entries=35671, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597971603 2020-11-17 07:26:12,862 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:26:20,776 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:26:23,932 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:26:36,585 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.40 MB heapSize=128.26 MB 2020-11-17 07:26:36,617 INFO [Thread-2419] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:26:37,242 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.40 MB at sequenceid=252056994 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/0538a974b8ab42588444d92312ca8e71 2020-11-17 07:26:37,248 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:26:37,268 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0538a974b8ab42588444d92312ca8e71, entries=312795, sequenceid=252056994, filesize=6.3 M 2020-11-17 07:26:37,277 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.40 MB/111573662, heapSize ~128.26 MB/134494536, currentSize=878.68 KB/899764 for 82421ce8c5c8b48c573baee83caa8305 in 692ms, sequenceid=252056994, compaction requested=true 2020-11-17 07:26:38,756 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 69721 ns. 2020-11-17 07:26:42,285 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:26:46,021 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:26:52,584 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:26:57,999 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597971603 with entries=15122, filesize=128.86 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598017908 2020-11-17 07:27:01,724 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:27:02,430 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.02 MB heapSize=128.33 MB 2020-11-17 07:27:02,463 INFO [Thread-2423] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:27:03,192 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.02 MB at sequenceid=273048898 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/1df482e2d18941179950bada908d4217 2020-11-17 07:27:03,217 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/1df482e2d18941179950bada908d4217, entries=278744, sequenceid=273048898, filesize=8.5 M 2020-11-17 07:27:03,222 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.02 MB/113264234, heapSize ~128.33 MB/134561696, currentSize=357.35 KB/365925 for fc93f6dcc564341dcc983bf30971f73d in 792ms, sequenceid=273048898, compaction requested=true 2020-11-17 07:27:03,223 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 07:27:03,223 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/4cd358d4b94b45a080ce413303beeea1, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/ba1b068ce8914983a93d2c23419dd9ff, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/1df482e2d18941179950bada908d4217] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=78.3 M 2020-11-17 07:27:03,262 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:27:03,268 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:27:03,303 INFO [Thread-2425] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:27:03,483 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:27:08,104 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:27:11,127 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:27:16,743 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:27:16,745 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:27:17,307 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.06 MB heapSize=128.40 MB 2020-11-17 07:27:17,338 INFO [Thread-2427] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:27:17,579 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:27:18,285 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.06 MB at sequenceid=263359006 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/f215beb37e6a430182a1c1362e7f380e 2020-11-17 07:27:18,308 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f215beb37e6a430182a1c1362e7f380e, entries=365015, sequenceid=263359006, filesize=8.0 M 2020-11-17 07:27:18,310 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.06 MB/107015175, heapSize ~128.40 MB/134639624, currentSize=1.03 MB/1075750 for e5dc5c620be19b02666a19c280ff582e in 1003ms, sequenceid=263359006, compaction requested=true 2020-11-17 07:27:19,969 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#530 average throughput is 58.11 MB/second, slept 19 time(s) and total slept time is 8957 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 07:27:20,059 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into e4de13d2666641d5b4e9ea60f6384e1d(size=78.7 M), total size for store is 33.4 G. This selection was in queue for 0sec, and took 16sec to execute. 2020-11-17 07:27:20,059 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=11, startTime=1605598023223; duration=16sec 2020-11-17 07:27:37,914 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.56 MB heapSize=128.02 MB 2020-11-17 07:27:37,963 INFO [Thread-2430] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:27:38,673 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.56 MB at sequenceid=260039212 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/8bfc4f06b0b2443496317cc40d55c4a3 2020-11-17 07:27:38,679 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:27:38,703 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/8bfc4f06b0b2443496317cc40d55c4a3, entries=372285, sequenceid=260039212, filesize=6.7 M 2020-11-17 07:27:38,704 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.56 MB/106490198, heapSize ~128.02 MB/134234512, currentSize=477.43 KB/488887 for 06f7ca9534710e37d90d9f8345864259 in 790ms, sequenceid=260039212, compaction requested=true 2020-11-17 07:27:39,650 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 45347 ns. 2020-11-17 07:27:42,000 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598017908 with entries=14206, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598061976 2020-11-17 07:27:45,024 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:00,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:02,490 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.18 MB heapSize=128.43 MB 2020-11-17 07:28:02,600 INFO [Thread-2433] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:03,195 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.18 MB at sequenceid=252059135 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/8ed1f793dd79436694b97fc4cbebd913 2020-11-17 07:28:03,219 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/8ed1f793dd79436694b97fc4cbebd913, entries=334253, sequenceid=252059135, filesize=6.0 M 2020-11-17 07:28:03,233 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.18 MB/110289682, heapSize ~128.43 MB/134671880, currentSize=961.78 KB/984865 for 82421ce8c5c8b48c573baee83caa8305 in 743ms, sequenceid=252059135, compaction requested=true 2020-11-17 07:28:03,234 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 07:28:03,234 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/549dbe1c90b343d4b9315c031a84bc0d, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0538a974b8ab42588444d92312ca8e71, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/8ed1f793dd79436694b97fc4cbebd913] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=31.4 M 2020-11-17 07:28:03,271 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:03,273 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:03,300 INFO [Thread-2435] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:06,029 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:13,321 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#534 average throughput is 52.80 MB/second, slept 9 time(s) and total slept time is 3973 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 07:28:13,397 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 92c2454f102744f698f22c3deaa1dd72(size=31.3 M), total size for store is 11.2 G. This selection was in queue for 0sec, and took 10sec to execute. 2020-11-17 07:28:13,398 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605598083233; duration=10sec 2020-11-17 07:28:19,364 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 29963 ns. 2020-11-17 07:28:19,364 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 8713 ns. 2020-11-17 07:28:52,336 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=87.88 MB heapSize=128.00 MB 2020-11-17 07:28:52,369 INFO [Thread-2439] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:53,349 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=87.88 MB at sequenceid=291578690 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/8d770bc50d15439793c8aa429c52f20c 2020-11-17 07:28:53,355 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:53,376 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/8d770bc50d15439793c8aa429c52f20c, entries=557003, sequenceid=291578690, filesize=8.6 M 2020-11-17 07:28:53,377 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~87.88 MB/92147360, heapSize ~128.00 MB/134218184, currentSize=59.29 KB/60709 for bdaa31a63b7f3d06d9909b7560333e79 in 1041ms, sequenceid=291578690, compaction requested=true 2020-11-17 07:28:53,378 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-17 07:28:53,378 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/f2ccc1a9025d488bb6ae6ef9e91d33a8, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/9590d29e15e140669096ddbcb3d1b79a, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/8d770bc50d15439793c8aa429c52f20c] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp, totalSize=44.6 M 2020-11-17 07:28:53,415 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:53,419 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:53,461 INFO [Thread-2441] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:57,565 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:59,013 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:59,015 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:59,108 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:28:59,615 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:29:00,885 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: bdaa31a63b7f3d06d9909b7560333e79#e#compaction#536 average throughput is 56.93 MB/second, slept 5 time(s) and total slept time is 1163 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 07:29:00,966 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of bdaa31a63b7f3d06d9909b7560333e79 into 82c0abc9c11e4b70b945d12514860a5b(size=44.0 M), total size for store is 31.8 G. This selection was in queue for 0sec, and took 7sec to execute. 2020-11-17 07:29:00,966 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., storeName=e, priority=10, startTime=1605598133377; duration=7sec 2020-11-17 07:29:10,446 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598061976 with entries=27424, filesize=128.43 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598150425 2020-11-17 07:29:10,447 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596264007 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605596264007 2020-11-17 07:29:10,454 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596370470 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605596370470 2020-11-17 07:29:10,462 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596447403 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605596447403 2020-11-17 07:29:10,468 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596550508 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605596550508 2020-11-17 07:29:10,477 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596651094 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605596651094 2020-11-17 07:29:23,960 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=170.48 MB, max=1.83 GB, blockCount=44252, accesses=23371586, hits=19942571, hitRatio=85.33%, , cachingAccesses=19994290, cachingHits=19937955, cachingHitsRatio=99.72%, evictions=2879, evicted=0, evictedPerRun=0.0 2020-11-17 07:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 07:29:39,377 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 125771 ns. 2020-11-17 07:29:39,377 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 101134 ns. 2020-11-17 07:29:39,377 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 249450 ns. 2020-11-17 07:29:55,248 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.29 MB heapSize=128.07 MB 2020-11-17 07:29:55,297 INFO [Thread-2445] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:29:56,223 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.29 MB at sequenceid=287723135 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/72efae330d5f41afa3ac82eeae7fdd5c 2020-11-17 07:29:56,230 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:29:56,247 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/72efae330d5f41afa3ac82eeae7fdd5c, entries=527989, sequenceid=287723135, filesize=9.0 M 2020-11-17 07:29:56,248 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.29 MB/94675848, heapSize ~128.07 MB/134286648, currentSize=68.92 KB/70574 for 5e5f05205d7e40b0141cd977dd009ba1 in 1000ms, sequenceid=287723135, compaction requested=true 2020-11-17 07:31:01,257 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598150425 with entries=38718, filesize=128.20 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598261227 2020-11-17 07:31:01,258 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596745116 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605596745116 2020-11-17 07:31:01,274 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596840954 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605596840954 2020-11-17 07:31:01,288 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605596932985 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605596932985 2020-11-17 07:31:01,294 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597039865 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605597039865 2020-11-17 07:31:01,299 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597122531 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605597122531 2020-11-17 07:31:01,306 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597228177 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605597228177 2020-11-17 07:31:01,312 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597300214 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605597300214 2020-11-17 07:31:01,317 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597420713 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605597420713 2020-11-17 07:31:01,322 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597489632 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605597489632 2020-11-17 07:31:28,797 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:31:32,832 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:31:36,089 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:31:46,101 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:31:50,980 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.71 MB heapSize=128.01 MB 2020-11-17 07:31:51,016 INFO [Thread-2452] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:31:51,809 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.71 MB at sequenceid=263371296 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/7cf4464c15b04145a8162c9c17e294e5 2020-11-17 07:31:51,814 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:31:51,833 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7cf4464c15b04145a8162c9c17e294e5, entries=379325, sequenceid=263371296, filesize=8.0 M 2020-11-17 07:31:51,836 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.71 MB/105598736, heapSize ~128.01 MB/134226216, currentSize=746.13 KB/764037 for e5dc5c620be19b02666a19c280ff582e in 857ms, sequenceid=263371296, compaction requested=true 2020-11-17 07:32:38,728 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 50308 ns. 2020-11-17 07:32:47,984 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598261227 with entries=36547, filesize=128.64 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598367950 2020-11-17 07:33:18,063 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:33:28,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18812 ns. 2020-11-17 07:33:28,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 31229 ns. 2020-11-17 07:33:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 46623 ns. 2020-11-17 07:33:38,859 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:34:02,503 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.88 MB heapSize=128.00 MB 2020-11-17 07:34:02,568 INFO [Thread-2459] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:34:03,556 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.88 MB at sequenceid=272167025 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/1a69fc1d4e514ba0803fcaa597a2af34 2020-11-17 07:34:03,562 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:34:03,582 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/1a69fc1d4e514ba0803fcaa597a2af34, entries=317788, sequenceid=272167025, filesize=8.6 M 2020-11-17 07:34:03,584 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.88 MB/109971609, heapSize ~128.00 MB/134218424, currentSize=47.67 KB/48819 for 0469ce862509756add4a8cdb00bc465c in 1081ms, sequenceid=272167025, compaction requested=true 2020-11-17 07:34:07,842 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:34:36,025 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.36 MB, max=1.83 GB, blockCount=44234, accesses=23633708, hits=20204675, hitRatio=85.49%, , cachingAccesses=20256412, cachingHits=20200059, cachingHitsRatio=99.72%, evictions=2909, evicted=0, evictedPerRun=0.0 2020-11-17 07:34:41,242 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598367950 with entries=37775, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598481203 2020-11-17 07:34:41,243 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597602270 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605597602270 2020-11-17 07:34:41,263 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597678780 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605597678780 2020-11-17 07:35:39,549 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.70 MB heapSize=128.04 MB 2020-11-17 07:35:39,565 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 121604 ns. 2020-11-17 07:35:39,566 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 215345 ns. 2020-11-17 07:35:39,566 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 118280 ns. 2020-11-17 07:35:39,593 INFO [Thread-2464] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:35:40,361 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.70 MB at sequenceid=263381679 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/1d04e2b78fbb426e92ff36a1bd083665 2020-11-17 07:35:40,370 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:35:40,389 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1d04e2b78fbb426e92ff36a1bd083665, entries=407943, sequenceid=263381679, filesize=8.0 M 2020-11-17 07:35:40,390 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.70 MB/103494607, heapSize ~128.04 MB/134255752, currentSize=502.06 KB/514110 for e5dc5c620be19b02666a19c280ff582e in 842ms, sequenceid=263381679, compaction requested=true 2020-11-17 07:35:40,392 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 07:35:40,392 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f215beb37e6a430182a1c1362e7f380e, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7cf4464c15b04145a8162c9c17e294e5, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1d04e2b78fbb426e92ff36a1bd083665] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.0 M 2020-11-17 07:35:40,434 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:35:40,436 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:35:40,463 INFO [Thread-2466] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:35:45,343 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#541 average throughput is 61.54 MB/second, slept 6 time(s) and total slept time is 1547 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 07:35:45,403 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 7286eda8f7fe4c51bff24283e83a000b(size=25.0 M), total size for store is 49.3 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 07:35:45,403 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605598540391; duration=5sec 2020-11-17 07:36:02,109 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.51 MB heapSize=128.25 MB 2020-11-17 07:36:02,156 INFO [Thread-2469] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:36:02,834 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.51 MB at sequenceid=252070990 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/6cad400610e741af8ea0f0d52c772113 2020-11-17 07:36:02,839 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:36:02,856 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/6cad400610e741af8ea0f0d52c772113, entries=324403, sequenceid=252070990, filesize=6.6 M 2020-11-17 07:36:02,859 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.51 MB/110640170, heapSize ~128.25 MB/134483408, currentSize=19.76 KB/20235 for 82421ce8c5c8b48c573baee83caa8305 in 750ms, sequenceid=252070990, compaction requested=true 2020-11-17 07:36:04,428 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598481203 with entries=27101, filesize=128.08 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598564405 2020-11-17 07:36:48,002 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:37:28,763 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598564405 with entries=30272, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598648728 2020-11-17 07:37:39,089 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 68010 ns. 2020-11-17 07:38:29,042 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 34661 ns. 2020-11-17 07:38:29,042 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 12701 ns. 2020-11-17 07:38:47,125 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598648728 with entries=26545, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598727101 2020-11-17 07:39:09,513 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=99.94 MB heapSize=128.00 MB 2020-11-17 07:39:09,554 INFO [Thread-2479] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:39:10,377 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.94 MB at sequenceid=260072624 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/9e54b8b13f674791adc775d114946257 2020-11-17 07:39:10,383 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:39:10,402 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/9e54b8b13f674791adc775d114946257, entries=386437, sequenceid=260072624, filesize=7.2 M 2020-11-17 07:39:10,403 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.94 MB/104790060, heapSize ~128.00 MB/134219008, currentSize=137.70 KB/141007 for 06f7ca9534710e37d90d9f8345864259 in 890ms, sequenceid=260072624, compaction requested=true 2020-11-17 07:39:10,405 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 07:39:10,405 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/1047b8dd542043fa8baef1b6d8220422, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/8bfc4f06b0b2443496317cc40d55c4a3, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/9e54b8b13f674791adc775d114946257] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=66.0 M 2020-11-17 07:39:10,442 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:39:10,449 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:39:10,474 INFO [Thread-2481] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:39:12,830 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.84 MB heapSize=128.33 MB 2020-11-17 07:39:12,867 INFO [Thread-2483] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:39:13,735 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.84 MB at sequenceid=263391816 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/22a10e2a8b6d4bf8be0a67f19b8c1ed1 2020-11-17 07:39:13,741 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:39:13,765 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/22a10e2a8b6d4bf8be0a67f19b8c1ed1, entries=367051, sequenceid=263391816, filesize=8.1 M 2020-11-17 07:39:13,767 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.84 MB/106786929, heapSize ~128.33 MB/134560512, currentSize=862.44 KB/883142 for e5dc5c620be19b02666a19c280ff582e in 937ms, sequenceid=263391816, compaction requested=true 2020-11-17 07:39:25,796 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#544 average throughput is 58.45 MB/second, slept 16 time(s) and total slept time is 5729 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 07:39:25,853 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:39:25,874 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 4e86dcdac5284730a735b95367386cdd(size=66.3 M), total size for store is 55.5 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 07:39:25,874 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=11, startTime=1605598750403; duration=15sec 2020-11-17 07:39:31,012 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.46 MB heapSize=128.04 MB 2020-11-17 07:39:31,037 INFO [Thread-2486] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:39:31,586 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.46 MB at sequenceid=252076830 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/4b6e4902975f42d3bde87c1dab7fd83a 2020-11-17 07:39:31,591 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:39:31,608 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/4b6e4902975f42d3bde87c1dab7fd83a, entries=308612, sequenceid=252076830, filesize=6.3 M 2020-11-17 07:39:31,609 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.46 MB/111629847, heapSize ~128.04 MB/134255344, currentSize=15.86 KB/16242 for 82421ce8c5c8b48c573baee83caa8305 in 597ms, sequenceid=252076830, compaction requested=true 2020-11-17 07:39:31,610 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 07:39:31,610 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/92c2454f102744f698f22c3deaa1dd72, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/6cad400610e741af8ea0f0d52c772113, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/4b6e4902975f42d3bde87c1dab7fd83a] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=44.2 M 2020-11-17 07:39:31,640 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:39:31,641 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:39:31,662 INFO [Thread-2488] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.78 MB, max=1.83 GB, blockCount=44225, accesses=23660110, hits=20207472, hitRatio=85.41%, , cachingAccesses=20259176, cachingHits=20202822, cachingHitsRatio=99.72%, evictions=2939, evicted=0, evictedPerRun=0.0 2020-11-17 07:39:38,757 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 51472 ns. 2020-11-17 07:39:44,966 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#547 average throughput is 55.71 MB/second, slept 14 time(s) and total slept time is 6107 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 07:39:45,039 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 95e75400e2544c91ba438c3e73a632c6(size=44.1 M), total size for store is 11.2 G. This selection was in queue for 0sec, and took 13sec to execute. 2020-11-17 07:39:45,039 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605598771610; duration=13sec 2020-11-17 07:39:49,614 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598727101 with entries=21081, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598789589 2020-11-17 07:41:26,668 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.99 MB heapSize=128.05 MB 2020-11-17 07:41:26,705 INFO [Thread-2494] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:41:27,894 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.99 MB at sequenceid=261920084 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/7a07929ffff7431b9c4dc60508539d2a 2020-11-17 07:41:27,901 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:41:27,923 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/7a07929ffff7431b9c4dc60508539d2a, entries=579448, sequenceid=261920084, filesize=8.9 M 2020-11-17 07:41:27,925 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.99 MB/90164761, heapSize ~128.05 MB/134271760, currentSize=131.89 KB/135060 for 73aae5715542e1a5370f7540d2117abc in 1258ms, sequenceid=261920084, compaction requested=true 2020-11-17 07:41:30,447 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598789589 with entries=35481, filesize=128.21 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598890426 2020-11-17 07:41:30,448 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597787344 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605597787344 2020-11-17 07:41:30,455 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597862820 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605597862820 2020-11-17 07:41:30,463 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605597971603 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605597971603 2020-11-17 07:41:38,766 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 146747 ns. 2020-11-17 07:41:38,767 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 138086 ns. 2020-11-17 07:41:38,767 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 193976 ns. 2020-11-17 07:42:04,923 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.87 MB heapSize=128.29 MB 2020-11-17 07:42:04,959 INFO [Thread-2498] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:42:05,688 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.87 MB at sequenceid=273072329 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/3a8ec01c48d84ac89822b3874b33bc52 2020-11-17 07:42:05,694 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:42:05,712 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/3a8ec01c48d84ac89822b3874b33bc52, entries=280350, sequenceid=273072329, filesize=8.4 M 2020-11-17 07:42:05,714 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~107.87 MB/113110930, heapSize ~128.29 MB/134521024, currentSize=16.35 KB/16744 for fc93f6dcc564341dcc983bf30971f73d in 791ms, sequenceid=273072329, compaction requested=true 2020-11-17 07:42:24,376 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.97 MB heapSize=128.47 MB 2020-11-17 07:42:24,413 INFO [Thread-2501] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:42:25,083 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.97 MB at sequenceid=263400869 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/18e1a9374b234ad080202ed225a83b42 2020-11-17 07:42:25,088 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:42:25,111 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/18e1a9374b234ad080202ed225a83b42, entries=353441, sequenceid=263400869, filesize=8.1 M 2020-11-17 07:42:25,113 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.97 MB/107968597, heapSize ~128.47 MB/134707864, currentSize=42.12 KB/43128 for e5dc5c620be19b02666a19c280ff582e in 737ms, sequenceid=263400869, compaction requested=true 2020-11-17 07:42:25,113 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 07:42:25,114 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7286eda8f7fe4c51bff24283e83a000b, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/22a10e2a8b6d4bf8be0a67f19b8c1ed1, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/18e1a9374b234ad080202ed225a83b42] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.2 M 2020-11-17 07:42:25,143 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:42:25,145 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:42:25,172 INFO [Thread-2503] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:42:33,357 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#551 average throughput is 61.72 MB/second, slept 10 time(s) and total slept time is 2846 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 07:42:33,440 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into d526c285b4414fcf81300d52ec2b5e53(size=42.5 M), total size for store is 49.3 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 07:42:33,440 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605598945113; duration=8sec 2020-11-17 07:43:00,991 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:43:18,063 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:43:23,374 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598890426 with entries=39559, filesize=128.31 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599003332 2020-11-17 07:43:23,374 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598017908 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605598017908 2020-11-17 07:43:29,063 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 12682 ns. 2020-11-17 07:43:29,063 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 31473 ns. 2020-11-17 07:43:38,829 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 60972 ns. 2020-11-17 07:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.54 MB, max=1.83 GB, blockCount=44200, accesses=23730885, hits=20262125, hitRatio=85.38%, , cachingAccesses=20313849, cachingHits=20257475, cachingHitsRatio=99.72%, evictions=2969, evicted=0, evictedPerRun=0.0 2020-11-17 07:44:39,403 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 35742 ns. 2020-11-17 07:44:57,377 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:45:12,096 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:45:22,307 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599003332 with entries=39126, filesize=128.60 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599122267 2020-11-17 07:45:22,328 WARN [Close-WAL-Writer-287] asyncfs.FanOutOneBlockAsyncDFSOutputHelper: complete file /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599003332 not finished, retry = 0 2020-11-17 07:46:02,364 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:46:20,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:46:43,417 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:46:52,015 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:46:55,288 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:46:58,363 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:47:01,617 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:47:05,604 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:47:10,206 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:47:25,843 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:47:28,429 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.90 MB heapSize=128.14 MB 2020-11-17 07:47:28,487 INFO [Thread-2518] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:47:29,353 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.90 MB at sequenceid=263414326 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/5b373065c4e04b0b935cc62b2ffe600d 2020-11-17 07:47:29,360 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:47:29,380 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5b373065c4e04b0b935cc62b2ffe600d, entries=407434, sequenceid=263414326, filesize=8.0 M 2020-11-17 07:47:29,381 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~98.90 MB/103709094, heapSize ~128.14 MB/134360456, currentSize=821.42 KB/841129 for e5dc5c620be19b02666a19c280ff582e in 952ms, sequenceid=263414326, compaction requested=true 2020-11-17 07:47:32,377 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.03 MB heapSize=128.00 MB 2020-11-17 07:47:32,408 INFO [Thread-2520] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:47:33,211 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.03 MB at sequenceid=272203837 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/0e116964d7a64933a13d2fdaa644cf33 2020-11-17 07:47:33,230 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:47:33,267 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/0e116964d7a64933a13d2fdaa644cf33, entries=315731, sequenceid=272203837, filesize=8.7 M 2020-11-17 07:47:33,269 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.03 MB/110133549, heapSize ~128.00 MB/134219240, currentSize=34.31 KB/35132 for 0469ce862509756add4a8cdb00bc465c in 893ms, sequenceid=272203837, compaction requested=true 2020-11-17 07:47:33,270 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 07:47:33,270 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/dbe3768d869b4062a2cc5af792658c37, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/e417228cc7ef4eb2b6509ac64d977982, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/1a69fc1d4e514ba0803fcaa597a2af34, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/0e116964d7a64933a13d2fdaa644cf33] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=215.7 M 2020-11-17 07:47:33,332 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:47:33,345 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:47:33,354 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:47:33,376 INFO [Thread-2522] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:47:34,972 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:47:39,097 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 264874 ns. 2020-11-17 07:47:39,097 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 110804 ns. 2020-11-17 07:47:39,098 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 111528 ns. 2020-11-17 07:47:42,666 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:47:49,197 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599122267 with entries=47068, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599269154 2020-11-17 07:47:58,066 INFO [DataStreamer for file /hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/23a73dea055c483f87faaafe60b6b92b] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:48:10,224 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:48:10,228 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:48:10,232 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:48:11,659 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:48:14,320 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#554 average throughput is 60.50 MB/second, slept 45 time(s) and total slept time is 16381 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 07:48:14,383 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 23a73dea055c483f87faaafe60b6b92b(size=215.3 M), total size for store is 42.8 G. This selection was in queue for 0sec, and took 41sec to execute. 2020-11-17 07:48:14,383 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605599253270; duration=41sec 2020-11-17 07:48:29,367 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18071 ns. 2020-11-17 07:48:29,367 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 39957 ns. 2020-11-17 07:48:39,237 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 73630 ns. 2020-11-17 07:48:40,728 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:49:03,024 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:49:10,401 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.28 MB, max=1.83 GB, blockCount=44231, accesses=24036997, hits=20528135, hitRatio=85.40%, , cachingAccesses=20579853, cachingHits=20523448, cachingHitsRatio=99.73%, evictions=2999, evicted=0, evictedPerRun=0.0 2020-11-17 07:50:06,162 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599269154 with entries=47103, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599406116 2020-11-17 07:50:39,183 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:50:39,258 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 43472 ns. 2020-11-17 07:51:12,200 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.40 MB heapSize=128.56 MB 2020-11-17 07:51:12,250 INFO [Thread-2532] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:51:13,030 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.40 MB at sequenceid=263424796 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/868f367a2e18418589145772b8762c35 2020-11-17 07:51:13,037 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:51:13,057 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/868f367a2e18418589145772b8762c35, entries=377303, sequenceid=263424796, filesize=8.3 M 2020-11-17 07:51:13,058 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.40 MB/106327972, heapSize ~128.56 MB/134803072, currentSize=154.52 KB/158228 for e5dc5c620be19b02666a19c280ff582e in 858ms, sequenceid=263424796, compaction requested=true 2020-11-17 07:51:13,059 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 07:51:13,060 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d526c285b4414fcf81300d52ec2b5e53, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5b373065c4e04b0b935cc62b2ffe600d, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/868f367a2e18418589145772b8762c35] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=58.8 M 2020-11-17 07:51:13,095 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:51:13,098 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:51:13,121 INFO [Thread-2534] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:51:24,658 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#556 average throughput is 61.15 MB/second, slept 12 time(s) and total slept time is 3984 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 07:51:24,726 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:51:24,748 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 4524ab0297654ee18777f2162f85f16e(size=60.3 M), total size for store is 49.3 G. This selection was in queue for 0sec, and took 11sec to execute. 2020-11-17 07:51:24,749 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605599473059; duration=11sec 2020-11-17 07:51:35,519 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:51:52,193 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599406116 with entries=33719, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599512172 2020-11-17 07:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 24430 ns. 2020-11-17 07:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 293471 ns. 2020-11-17 07:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 13510 ns. 2020-11-17 07:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 92939 ns. 2020-11-17 07:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 69985 ns. 2020-11-17 07:54:09,463 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599512172 with entries=46632, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599649422 2020-11-17 07:54:23,594 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.43 MB heapSize=128.00 MB 2020-11-17 07:54:23,626 INFO [Thread-2545] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:54:24,695 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.43 MB at sequenceid=287797141 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/83888256fefb49bc85ad09e3046371e2 2020-11-17 07:54:24,701 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:54:24,717 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/83888256fefb49bc85ad09e3046371e2, entries=524646, sequenceid=287797141, filesize=9.1 M 2020-11-17 07:54:24,719 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.43 MB/94821858, heapSize ~128.00 MB/134219784, currentSize=35.55 KB/36405 for 5e5f05205d7e40b0141cd977dd009ba1 in 1125ms, sequenceid=287797141, compaction requested=true 2020-11-17 07:54:25,638 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.56 MB heapSize=128.65 MB 2020-11-17 07:54:25,667 INFO [Thread-2547] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:54:26,363 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.56 MB at sequenceid=263433593 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/1b0899095b73437d9b6861573a571fbd 2020-11-17 07:54:26,389 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1b0899095b73437d9b6861573a571fbd, entries=347972, sequenceid=263433593, filesize=8.2 M 2020-11-17 07:54:26,390 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~103.56 MB/108595227, heapSize ~128.65 MB/134899952, currentSize=830.31 KB/850235 for e5dc5c620be19b02666a19c280ff582e in 752ms, sequenceid=263433593, compaction requested=true 2020-11-17 07:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.48 MB, max=1.83 GB, blockCount=44183, accesses=24051010, hits=20530669, hitRatio=85.36%, , cachingAccesses=20582384, cachingHits=20525975, cachingHitsRatio=99.73%, evictions=3029, evicted=0, evictedPerRun=0.0 2020-11-17 07:54:39,301 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 50547 ns. 2020-11-17 07:56:18,243 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:56:28,285 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599649422 with entries=47926, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599788222 2020-11-17 07:56:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 48088 ns. 2020-11-17 07:56:48,004 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:57:00,241 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:57:30,406 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.80 MB heapSize=128.13 MB 2020-11-17 07:57:30,443 INFO [Thread-2558] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:57:31,526 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.80 MB at sequenceid=261966560 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/af2c242208ac4bd180804f551c5f69fc 2020-11-17 07:57:31,531 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:57:31,554 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/af2c242208ac4bd180804f551c5f69fc, entries=579145, sequenceid=261966560, filesize=9.1 M 2020-11-17 07:57:31,556 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.80 MB/89972446, heapSize ~128.14 MB/134359376, currentSize=173.01 KB/177162 for 73aae5715542e1a5370f7540d2117abc in 1150ms, sequenceid=261966560, compaction requested=true 2020-11-17 07:57:31,558 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 07:57:31,558 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/21f7ec74002149bd8f49b23fea1169b2, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/7a07929ffff7431b9c4dc60508539d2a, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/af2c242208ac4bd180804f551c5f69fc] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=63.2 M 2020-11-17 07:57:31,605 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:57:31,610 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:57:31,648 INFO [Thread-2560] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:57:39,234 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:57:41,169 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#560 average throughput is 61.26 MB/second, slept 7 time(s) and total slept time is 1169 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 07:57:41,226 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 2ab60e7e1f44419e99ce73547f03b85a(size=63.7 M), total size for store is 12.6 G. This selection was in queue for 0sec, and took 9sec to execute. 2020-11-17 07:57:41,226 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=10, startTime=1605599851556; duration=9sec 2020-11-17 07:57:55,650 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.46 MB heapSize=128.54 MB 2020-11-17 07:57:55,689 INFO [Thread-2563] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:57:56,449 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.46 MB at sequenceid=273096917 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/bb460a232986427ebcdef0af149238f4 2020-11-17 07:57:56,455 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:57:56,471 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/bb460a232986427ebcdef0af149238f4, entries=275165, sequenceid=273096917, filesize=8.5 M 2020-11-17 07:57:56,473 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.46 MB/113728582, heapSize ~128.54 MB/134781440, currentSize=600.09 KB/614496 for fc93f6dcc564341dcc983bf30971f73d in 823ms, sequenceid=273096917, compaction requested=true 2020-11-17 07:57:56,473 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 07:57:56,474 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/e4de13d2666641d5b4e9ea60f6384e1d, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/3a8ec01c48d84ac89822b3874b33bc52, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/bb460a232986427ebcdef0af149238f4] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=95.7 M 2020-11-17 07:57:56,501 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:57:56,506 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:57:56,529 INFO [Thread-2565] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:57:58,077 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:10,197 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:12,049 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:13,260 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:13,865 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:15,505 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#562 average throughput is 62.23 MB/second, slept 23 time(s) and total slept time is 9445 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 07:58:15,571 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:15,591 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into 30625b3021ca484986c7a2bc29e77ce5(size=95.9 M), total size for store is 33.4 G. This selection was in queue for 0sec, and took 19sec to execute. 2020-11-17 07:58:15,591 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=11, startTime=1605599876473; duration=19sec 2020-11-17 07:58:23,845 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:26,949 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599788222 with entries=43159, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599906898 2020-11-17 07:58:27,409 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:33,358 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:38,716 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 31518 ns. 2020-11-17 07:58:38,716 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 12646 ns. 2020-11-17 07:58:42,204 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:49,724 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:56,079 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.53 MB heapSize=128.52 MB 2020-11-17 07:58:56,131 INFO [Thread-2571] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:56,879 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.53 MB at sequenceid=263446607 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/374c7ec6ba754c64bcab238193034c8c 2020-11-17 07:58:56,886 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:56,904 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/374c7ec6ba754c64bcab238193034c8c, entries=375309, sequenceid=263446607, filesize=8.2 M 2020-11-17 07:58:56,906 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.53 MB/106464208, heapSize ~128.52 MB/134761104, currentSize=708.01 KB/725001 for e5dc5c620be19b02666a19c280ff582e in 827ms, sequenceid=263446607, compaction requested=true 2020-11-17 07:58:56,907 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 07:58:56,907 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/0e4e88984b85473097cbad43fa3340c6, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ebefc79f6f4d4df0aa1bbad6d5362973, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/4524ab0297654ee18777f2162f85f16e, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1b0899095b73437d9b6861573a571fbd, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/374c7ec6ba754c64bcab238193034c8c] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=408.8 M 2020-11-17 07:58:56,962 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:56,966 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:56,969 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:56,970 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:58:56,996 INFO [Thread-2573] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:59:00,163 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:59:06,828 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:59:12,402 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:59:19,441 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:59:20,285 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a204e2c26acf42d2af60e900f55b5304] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=170.21 MB, max=1.83 GB, blockCount=44253, accesses=24367415, hits=20780059, hitRatio=85.28%, , cachingAccesses=20831805, cachingHits=20775325, cachingHitsRatio=99.73%, evictions=3059, evicted=0, evictedPerRun=0.0 2020-11-17 07:59:36,106 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:59:38,811 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 95300 ns. 2020-11-17 07:59:38,811 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 282878 ns. 2020-11-17 07:59:38,811 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 99996 ns. 2020-11-17 07:59:41,304 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:59:46,080 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a204e2c26acf42d2af60e900f55b5304] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:59:49,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:59:53,560 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 07:59:59,295 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:00:02,336 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:00:06,433 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=97.16 MB heapSize=128.05 MB 2020-11-17 08:00:06,468 INFO [Thread-2581] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:00:07,342 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.16 MB at sequenceid=260133487 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/6062e318d37149f8ad1c8f6d45a4e286 2020-11-17 08:00:07,373 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/6062e318d37149f8ad1c8f6d45a4e286, entries=418451, sequenceid=260133487, filesize=8.1 M 2020-11-17 08:00:07,374 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~97.16 MB/101881027, heapSize ~128.05 MB/134268048, currentSize=45.85 KB/46947 for 06f7ca9534710e37d90d9f8345864259 in 941ms, sequenceid=260133487, compaction requested=true 2020-11-17 08:00:10,450 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a204e2c26acf42d2af60e900f55b5304] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:00:12,371 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:00:12,911 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:00:14,544 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:00:14,548 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:00:14,552 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:00:14,829 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:00:15,439 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#564 average throughput is 60.29 MB/second, slept 83 time(s) and total slept time is 26028 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 08:00:15,519 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 5 file(s) in e of e5dc5c620be19b02666a19c280ff582e into a204e2c26acf42d2af60e900f55b5304(size=412.7 M), total size for store is 49.4 G. This selection was in queue for 0sec, and took 1mins, 18sec to execute. 2020-11-17 08:00:15,519 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605599936906; duration=1mins, 18sec 2020-11-17 08:00:19,510 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599906898 with entries=39954, filesize=128.24 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600019483 2020-11-17 08:00:21,669 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:00:37,550 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:00:38,716 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 50345 ns. 2020-11-17 08:00:44,351 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:01:13,082 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:01:13,212 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.44 MB heapSize=128.64 MB 2020-11-17 08:01:13,253 INFO [Thread-2586] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:01:14,031 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.44 MB at sequenceid=272241985 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/8a38ba30c01a4efc9d9d33f502eba9b5 2020-11-17 08:01:14,052 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/8a38ba30c01a4efc9d9d33f502eba9b5, entries=318982, sequenceid=272241985, filesize=8.7 M 2020-11-17 08:01:14,054 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.44 MB/110561951, heapSize ~128.64 MB/134892912, currentSize=30.29 KB/31021 for 0469ce862509756add4a8cdb00bc465c in 842ms, sequenceid=272241985, compaction requested=true 2020-11-17 08:01:19,934 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=103.33 MB heapSize=128.00 MB 2020-11-17 08:01:19,970 INFO [Thread-2588] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:01:20,818 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.33 MB at sequenceid=252110302 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/bde7d36c41054110a4f92ab3543747e7 2020-11-17 08:01:20,824 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:01:20,842 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/bde7d36c41054110a4f92ab3543747e7, entries=350196, sequenceid=252110302, filesize=7.2 M 2020-11-17 08:01:20,844 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.33 MB/108344815, heapSize ~128.00 MB/134222304, currentSize=18.72 KB/19169 for 82421ce8c5c8b48c573baee83caa8305 in 911ms, sequenceid=252110302, compaction requested=true 2020-11-17 08:01:31,970 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:01:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 45352 ns. 2020-11-17 08:02:06,436 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:02:15,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:02:18,386 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600019483 with entries=40524, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600138345 2020-11-17 08:02:22,970 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:02:31,633 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:02:39,558 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.92 MB heapSize=128.00 MB 2020-11-17 08:02:39,609 INFO [Thread-2594] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:02:40,949 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.92 MB at sequenceid=291677217 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/2bc9049501824e078eda9df6f89c6f0e 2020-11-17 08:02:40,958 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:02:40,986 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/2bc9049501824e078eda9df6f89c6f0e, entries=543529, sequenceid=291677217, filesize=9.0 M 2020-11-17 08:02:40,994 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~88.92 MB/93240569, heapSize ~128.00 MB/134218536, currentSize=91.90 KB/94101 for bdaa31a63b7f3d06d9909b7560333e79 in 1435ms, sequenceid=291677217, compaction requested=true 2020-11-17 08:02:41,565 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.96 MB heapSize=128.24 MB 2020-11-17 08:02:41,593 INFO [Thread-2596] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:02:42,644 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.96 MB at sequenceid=263457284 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a9fcb2ecac4c4e859e8ec9c686d520ab 2020-11-17 08:02:42,674 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a9fcb2ecac4c4e859e8ec9c686d520ab, entries=407312, sequenceid=263457284, filesize=8.1 M 2020-11-17 08:02:42,677 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.96 MB/103771759, heapSize ~128.24 MB/134469648, currentSize=754.90 KB/773021 for e5dc5c620be19b02666a19c280ff582e in 1112ms, sequenceid=263457284, compaction requested=true 2020-11-17 08:02:56,848 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:03:06,386 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:03:22,368 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:03:38,752 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 16045 ns. 2020-11-17 08:03:38,752 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 38183 ns. 2020-11-17 08:03:41,665 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:03:45,185 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600138345 with entries=29740, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600225159 2020-11-17 08:03:45,186 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598061976 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605598061976 2020-11-17 08:03:45,195 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598150425 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605598150425 2020-11-17 08:03:45,201 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598261227 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605598261227 2020-11-17 08:03:45,208 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598367950 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605598367950 2020-11-17 08:03:45,213 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598481203 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605598481203 2020-11-17 08:03:45,218 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598564405 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605598564405 2020-11-17 08:03:45,223 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598648728 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605598648728 2020-11-17 08:03:45,227 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598727101 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605598727101 2020-11-17 08:03:45,232 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598789589 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605598789589 2020-11-17 08:03:45,237 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605598890426 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605598890426 2020-11-17 08:03:45,242 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599003332 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605599003332 2020-11-17 08:03:45,246 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599122267 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605599122267 2020-11-17 08:03:45,250 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599269154 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605599269154 2020-11-17 08:03:45,255 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599406116 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605599406116 2020-11-17 08:03:45,260 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599512172 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605599512172 2020-11-17 08:04:29,594 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.88 MB, max=1.83 GB, blockCount=44208, accesses=25067702, hits=21441719, hitRatio=85.54%, , cachingAccesses=21493481, cachingHits=21436933, cachingHitsRatio=99.74%, evictions=3089, evicted=0, evictedPerRun=0.0 2020-11-17 08:04:36,768 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:05:10,357 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:05:27,000 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600225159 with entries=32556, filesize=128.09 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600326973 2020-11-17 08:05:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 135103 ns. 2020-11-17 08:05:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 263758 ns. 2020-11-17 08:05:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 83616 ns. 2020-11-17 08:06:33,792 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.82 MB heapSize=128.46 MB 2020-11-17 08:06:33,826 INFO [Thread-2607] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:06:34,512 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.82 MB at sequenceid=263467527 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/ec0d5ffd5eb043d586c7eeb3713d13f0 2020-11-17 08:06:34,519 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:06:34,545 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ec0d5ffd5eb043d586c7eeb3713d13f0, entries=355645, sequenceid=263467527, filesize=8.3 M 2020-11-17 08:06:34,560 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.82 MB/107810356, heapSize ~128.46 MB/134702328, currentSize=525.54 KB/538156 for e5dc5c620be19b02666a19c280ff582e in 768ms, sequenceid=263467527, compaction requested=true 2020-11-17 08:06:39,737 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 39906 ns. 2020-11-17 08:06:39,737 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 47752 ns. 2020-11-17 08:06:45,605 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600326973 with entries=24957, filesize=128.39 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600405584 2020-11-17 08:06:48,004 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:06:53,004 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.59 MB heapSize=128.08 MB 2020-11-17 08:06:53,029 INFO [Thread-2610] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:06:53,810 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.59 MB at sequenceid=252118672 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/c9247a6c0f964961a5c4f35e25f4d790 2020-11-17 08:06:53,817 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:06:53,833 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/c9247a6c0f964961a5c4f35e25f4d790, entries=307574, sequenceid=252118672, filesize=6.4 M 2020-11-17 08:06:53,835 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.59 MB/111765461, heapSize ~128.08 MB/134299592, currentSize=260.37 KB/266614 for 82421ce8c5c8b48c573baee83caa8305 in 831ms, sequenceid=252118672, compaction requested=true 2020-11-17 08:06:53,835 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 08:06:53,835 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/95e75400e2544c91ba438c3e73a632c6, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/bde7d36c41054110a4f92ab3543747e7, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/c9247a6c0f964961a5c4f35e25f4d790] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=57.7 M 2020-11-17 08:06:53,868 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:06:53,870 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:06:53,894 INFO [Thread-2612] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:07:10,367 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#572 average throughput is 57.73 MB/second, slept 19 time(s) and total slept time is 7793 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:07:10,523 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 2a368c40b6464245974cc3ae2b8ad01f(size=57.7 M), total size for store is 11.3 G. This selection was in queue for 0sec, and took 16sec to execute. 2020-11-17 08:07:10,523 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605600413835; duration=16sec 2020-11-17 08:08:39,084 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 45421 ns. 2020-11-17 08:08:39,085 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 16975 ns. 2020-11-17 08:08:45,530 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600405584 with entries=41138, filesize=128.16 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600525499 2020-11-17 08:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.99 MB, max=1.83 GB, blockCount=44207, accesses=25135891, hits=21494478, hitRatio=85.51%, , cachingAccesses=21546238, cachingHits=21489687, cachingHitsRatio=99.74%, evictions=3119, evicted=0, evictedPerRun=0.0 2020-11-17 08:10:02,625 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600525499 with entries=26125, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600602588 2020-11-17 08:10:32,315 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.48 MB heapSize=128.00 MB 2020-11-17 08:10:32,365 INFO [Thread-2623] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:10:33,060 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.48 MB at sequenceid=263478568 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/3ff3cdd6bbbd49209d72f025255f50ad 2020-11-17 08:10:33,066 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:10:33,088 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3ff3cdd6bbbd49209d72f025255f50ad, entries=354017, sequenceid=263478568, filesize=8.1 M 2020-11-17 08:10:33,089 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.48 MB/107457217, heapSize ~128.00 MB/134220456, currentSize=21.72 KB/22240 for e5dc5c620be19b02666a19c280ff582e in 774ms, sequenceid=263478568, compaction requested=true 2020-11-17 08:10:33,091 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 08:10:33,091 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a9fcb2ecac4c4e859e8ec9c686d520ab, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ec0d5ffd5eb043d586c7eeb3713d13f0, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3ff3cdd6bbbd49209d72f025255f50ad] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.5 M 2020-11-17 08:10:33,118 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:10:33,120 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:10:33,145 INFO [Thread-2625] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:10:38,407 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#574 average throughput is 57.62 MB/second, slept 5 time(s) and total slept time is 1877 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:10:38,474 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 7903e619c126477fbfa06dc91fab3199(size=25.5 M), total size for store is 49.4 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 08:10:38,474 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605600633090; duration=5sec 2020-11-17 08:10:55,678 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:11:06,030 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:11:39,124 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 215499 ns. 2020-11-17 08:11:39,124 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 97808 ns. 2020-11-17 08:11:39,124 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 83895 ns. 2020-11-17 08:11:54,592 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600602588 with entries=35459, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600714554 2020-11-17 08:12:39,104 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 66204 ns. 2020-11-17 08:12:39,105 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 48493 ns. 2020-11-17 08:12:51,387 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.60 MB heapSize=128.01 MB 2020-11-17 08:12:51,426 INFO [Thread-2633] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:12:52,171 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.60 MB at sequenceid=260169528 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/9c552c70639d4929aa44ded872f982f3 2020-11-17 08:12:52,178 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:12:52,195 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/9c552c70639d4929aa44ded872f982f3, entries=381024, sequenceid=260169528, filesize=7.2 M 2020-11-17 08:12:52,197 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.60 MB/105490447, heapSize ~128.01 MB/134223208, currentSize=257.54 KB/263720 for 06f7ca9534710e37d90d9f8345864259 in 810ms, sequenceid=260169528, compaction requested=true 2020-11-17 08:12:52,199 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 08:12:52,199 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/4e86dcdac5284730a735b95367386cdd, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/6062e318d37149f8ad1c8f6d45a4e286, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/9c552c70639d4929aa44ded872f982f3] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=81.7 M 2020-11-17 08:12:52,252 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:12:52,256 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:12:52,279 INFO [Thread-2635] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:12:58,550 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.71 MB heapSize=128.57 MB 2020-11-17 08:12:58,582 INFO [Thread-2637] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:12:59,290 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.71 MB at sequenceid=252127580 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/b568f6c7fb764251aadf8ab472627dd8 2020-11-17 08:12:59,296 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:12:59,317 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b568f6c7fb764251aadf8ab472627dd8, entries=312510, sequenceid=252127580, filesize=6.4 M 2020-11-17 08:12:59,318 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.71 MB/111898046, heapSize ~128.57 MB/134816872, currentSize=50.56 KB/51772 for 82421ce8c5c8b48c573baee83caa8305 in 768ms, sequenceid=252127580, compaction requested=true 2020-11-17 08:13:06,175 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600714554 with entries=20800, filesize=128.43 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600786152 2020-11-17 08:13:11,062 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#576 average throughput is 58.12 MB/second, slept 20 time(s) and total slept time is 7806 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:13:11,116 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 90a7ee8828c34b04addff0d4aa3c9f37(size=82.0 M), total size for store is 55.5 G. This selection was in queue for 0sec, and took 18sec to execute. 2020-11-17 08:13:11,117 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=11, startTime=1605600772197; duration=18sec 2020-11-17 08:13:18,084 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:13:32,905 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:13:36,286 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.36 MB heapSize=128.61 MB 2020-11-17 08:13:36,327 INFO [Thread-2640] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:13:37,025 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.36 MB at sequenceid=273120640 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/5804e2f879184cf9bbf986cf9a3fbb7f 2020-11-17 08:13:37,030 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:13:37,050 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/5804e2f879184cf9bbf986cf9a3fbb7f, entries=277828, sequenceid=273120640, filesize=8.5 M 2020-11-17 08:13:37,052 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.36 MB/113627251, heapSize ~128.61 MB/134859960, currentSize=81.21 KB/83164 for fc93f6dcc564341dcc983bf30971f73d in 766ms, sequenceid=273120640, compaction requested=true 2020-11-17 08:13:43,000 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:13:49,174 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 44973 ns. 2020-11-17 08:13:49,174 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 10828 ns. 2020-11-17 08:13:58,529 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:14:04,136 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:14:11,021 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:14:14,179 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:14:18,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.55 MB, max=1.83 GB, blockCount=44193, accesses=25170207, hits=21506061, hitRatio=85.44%, , cachingAccesses=21557831, cachingHits=21501240, cachingHitsRatio=99.74%, evictions=3149, evicted=0, evictedPerRun=0.0 2020-11-17 08:14:38,000 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.52 MB heapSize=128.00 MB 2020-11-17 08:14:38,043 INFO [Thread-2644] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:14:39,122 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.52 MB at sequenceid=262014898 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/54b2328cc63040a39a57bf4962cf48b5 2020-11-17 08:14:39,131 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:14:39,149 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/54b2328cc63040a39a57bf4962cf48b5, entries=579898, sequenceid=262014898, filesize=8.9 M 2020-11-17 08:14:39,151 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.52 MB/89670905, heapSize ~128.00 MB/134217832, currentSize=65.45 KB/67017 for 73aae5715542e1a5370f7540d2117abc in 1151ms, sequenceid=262014898, compaction requested=true 2020-11-17 08:14:44,230 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:14:55,431 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.29 MB heapSize=128.00 MB 2020-11-17 08:14:55,471 INFO [Thread-2647] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:14:56,225 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.29 MB at sequenceid=263489412 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/724d897fec6145c18c3c89f7d3f6ea6c 2020-11-17 08:14:56,232 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:14:56,259 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/724d897fec6145c18c3c89f7d3f6ea6c, entries=412679, sequenceid=263489412, filesize=8.0 M 2020-11-17 08:14:56,261 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.29 MB/103061270, heapSize ~128.00 MB/134219160, currentSize=353.96 KB/362460 for e5dc5c620be19b02666a19c280ff582e in 830ms, sequenceid=263489412, compaction requested=true 2020-11-17 08:15:00,113 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600786152 with entries=34501, filesize=128.57 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600900093 2020-11-17 08:15:06,207 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.98 MB heapSize=128.23 MB 2020-11-17 08:15:06,232 INFO [Thread-2649] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:15:06,991 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.98 MB at sequenceid=272278085 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/74d2dbf7b63a49f3a186fa37fed61574 2020-11-17 08:15:06,999 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:15:07,018 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/74d2dbf7b63a49f3a186fa37fed61574, entries=319654, sequenceid=272278085, filesize=8.7 M 2020-11-17 08:15:07,022 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.98 MB/110083657, heapSize ~128.23 MB/134455464, currentSize=825.92 KB/845739 for 0469ce862509756add4a8cdb00bc465c in 815ms, sequenceid=272278085, compaction requested=true 2020-11-17 08:15:11,469 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:15:21,497 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:16:10,228 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:16:15,503 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600900093 with entries=24297, filesize=128.33 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600975468 2020-11-17 08:17:37,605 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:17:38,735 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 241080 ns. 2020-11-17 08:17:38,736 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 185550 ns. 2020-11-17 08:17:38,736 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 75442 ns. 2020-11-17 08:17:43,527 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:17:49,381 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:02,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:02,624 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.54 MB heapSize=128.01 MB 2020-11-17 08:18:02,680 INFO [Thread-2658] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:02,709 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600975468 with entries=32559, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601082666 2020-11-17 08:18:03,846 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.55 MB at sequenceid=287866969 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/66d57068ab084a46aba3dfe20926e2b2 2020-11-17 08:18:03,875 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/66d57068ab084a46aba3dfe20926e2b2, entries=521487, sequenceid=287866969, filesize=9.1 M 2020-11-17 08:18:03,876 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.55 MB/94943959, heapSize ~128.01 MB/134224576, currentSize=56.25 KB/57598 for 5e5f05205d7e40b0141cd977dd009ba1 in 1252ms, sequenceid=287866969, compaction requested=true 2020-11-17 08:18:03,877 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 08:18:03,877 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/72efae330d5f41afa3ac82eeae7fdd5c, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/83888256fefb49bc85ad09e3046371e2, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/66d57068ab084a46aba3dfe20926e2b2] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=27.1 M 2020-11-17 08:18:03,917 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:03,924 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:03,953 INFO [Thread-2660] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:08,590 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#583 average throughput is 58.12 MB/second, slept 5 time(s) and total slept time is 1134 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:18:08,682 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into 60ca2181edae4ee081597535d8e8d628(size=27.5 M), total size for store is 24.3 G. This selection was in queue for 0sec, and took 4sec to execute. 2020-11-17 08:18:08,682 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=10, startTime=1605601083876; duration=4sec 2020-11-17 08:18:22,518 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.39 MB heapSize=128.29 MB 2020-11-17 08:18:22,577 INFO [Thread-2662] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:23,280 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.39 MB at sequenceid=252134602 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/a9914a708ad34b30baad533eda025e4d 2020-11-17 08:18:23,287 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:23,304 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a9914a708ad34b30baad533eda025e4d, entries=311655, sequenceid=252134602, filesize=6.4 M 2020-11-17 08:18:23,305 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.39 MB/111557234, heapSize ~128.29 MB/134524056, currentSize=627.07 KB/642118 for 82421ce8c5c8b48c573baee83caa8305 in 787ms, sequenceid=252134602, compaction requested=true 2020-11-17 08:18:23,306 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 08:18:23,306 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/2a368c40b6464245974cc3ae2b8ad01f, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b568f6c7fb764251aadf8ab472627dd8, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a9914a708ad34b30baad533eda025e4d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=70.4 M 2020-11-17 08:18:23,353 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:23,357 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:23,383 INFO [Thread-2664] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:35,184 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:39,184 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 76136 ns. 2020-11-17 08:18:39,184 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 55768 ns. 2020-11-17 08:18:42,019 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.33 MB heapSize=128.07 MB 2020-11-17 08:18:42,091 INFO [Thread-2667] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:42,949 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.34 MB at sequenceid=263499111 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/eef28afa2aee4e3e81d371bc2f1c27a0 2020-11-17 08:18:42,964 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:43,025 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/eef28afa2aee4e3e81d371bc2f1c27a0, entries=370199, sequenceid=263499111, filesize=8.0 M 2020-11-17 08:18:43,030 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.34 MB/106260426, heapSize ~128.07 MB/134293728, currentSize=1.27 MB/1334336 for e5dc5c620be19b02666a19c280ff582e in 1011ms, sequenceid=263499111, compaction requested=true 2020-11-17 08:18:43,031 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 08:18:43,031 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7903e619c126477fbfa06dc91fab3199, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/724d897fec6145c18c3c89f7d3f6ea6c, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/eef28afa2aee4e3e81d371bc2f1c27a0] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.4 M 2020-11-17 08:18:43,072 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:43,074 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:43,098 INFO [Thread-2669] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:43,641 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#585 average throughput is 57.45 MB/second, slept 22 time(s) and total slept time is 8858 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 08:18:43,708 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into b5035d1014a849bd88807e0eca4109d6(size=70.4 M), total size for store is 11.3 G. This selection was in queue for 0sec, and took 20sec to execute. 2020-11-17 08:18:43,708 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605601103306; duration=20sec 2020-11-17 08:18:51,347 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#587 average throughput is 60.96 MB/second, slept 9 time(s) and total slept time is 2030 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 08:18:51,446 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 37f89b84649b474c95c3ad3d56d7dbfb(size=42.7 M), total size for store is 49.4 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 08:18:51,446 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605601123030; duration=8sec 2020-11-17 08:18:56,120 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:18:58,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 17333 ns. 2020-11-17 08:18:58,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 31964 ns. 2020-11-17 08:19:28,761 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601082666 with entries=26377, filesize=128.37 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601168707 2020-11-17 08:19:28,762 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599649422 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605599649422 2020-11-17 08:19:28,775 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599788222 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605599788222 2020-11-17 08:19:28,781 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605599906898 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605599906898 2020-11-17 08:19:28,787 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600019483 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605600019483 2020-11-17 08:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.45 MB, max=1.83 GB, blockCount=44221, accesses=25352108, hits=21656445, hitRatio=85.42%, , cachingAccesses=21708224, cachingHits=21651604, cachingHitsRatio=99.74%, evictions=3179, evicted=0, evictedPerRun=0.0 2020-11-17 08:19:57,983 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:20:02,407 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:21:16,243 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601168707 with entries=32956, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601276190 2020-11-17 08:22:24,743 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.31 MB heapSize=128.38 MB 2020-11-17 08:22:24,789 INFO [Thread-2683] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:22:25,519 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.31 MB at sequenceid=263508458 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/373e8c032fd141d88f0f6825b444cd66 2020-11-17 08:22:25,525 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:22:25,543 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/373e8c032fd141d88f0f6825b444cd66, entries=361453, sequenceid=263508458, filesize=8.1 M 2020-11-17 08:22:25,544 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.31 MB/107281981, heapSize ~128.38 MB/134617656, currentSize=478.33 KB/489808 for e5dc5c620be19b02666a19c280ff582e in 801ms, sequenceid=263508458, compaction requested=true 2020-11-17 08:22:32,745 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601276190 with entries=22681, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601352723 2020-11-17 08:22:37,647 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:23:13,752 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:23:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 86849 ns. 2020-11-17 08:23:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 199175 ns. 2020-11-17 08:23:38,750 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 294931 ns. 2020-11-17 08:23:58,062 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:23:58,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 12826 ns. 2020-11-17 08:23:58,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 27909 ns. 2020-11-17 08:23:59,908 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.74 MB heapSize=128.01 MB 2020-11-17 08:23:59,947 INFO [Thread-2689] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:24:00,695 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.74 MB at sequenceid=260198923 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/0236f1f9b5114fa8aa51a9f77cccd6a7 2020-11-17 08:24:00,718 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/0236f1f9b5114fa8aa51a9f77cccd6a7, entries=375930, sequenceid=260198923, filesize=7.1 M 2020-11-17 08:24:00,720 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.74 MB/105635834, heapSize ~128.01 MB/134229808, currentSize=125.74 KB/128754 for 06f7ca9534710e37d90d9f8345864259 in 812ms, sequenceid=260198923, compaction requested=true 2020-11-17 08:24:03,011 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.18 MB heapSize=128.04 MB 2020-11-17 08:24:03,033 INFO [Thread-2691] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:24:03,618 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.18 MB at sequenceid=252141848 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/2ae3ae70955a4f70bddc8f02fc6b0cbb 2020-11-17 08:24:03,639 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/2ae3ae70955a4f70bddc8f02fc6b0cbb, entries=312472, sequenceid=252141848, filesize=6.3 M 2020-11-17 08:24:03,640 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.18 MB/111332872, heapSize ~128.04 MB/134260008, currentSize=10.83 KB/11092 for 82421ce8c5c8b48c573baee83caa8305 in 629ms, sequenceid=252141848, compaction requested=true 2020-11-17 08:24:13,638 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601352723 with entries=30529, filesize=128.48 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601453613 2020-11-17 08:24:24,933 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:24:31,554 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:24:35,904 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.19 MB, max=1.83 GB, blockCount=44201, accesses=25484319, hits=21788646, hitRatio=85.50%, , cachingAccesses=21840435, cachingHits=21783805, cachingHitsRatio=99.74%, evictions=3209, evicted=0, evictedPerRun=0.0 2020-11-17 08:24:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 66110 ns. 2020-11-17 08:24:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 40861 ns. 2020-11-17 08:25:37,525 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601453613 with entries=25181, filesize=128.59 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601537479 2020-11-17 08:26:21,720 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:26:48,014 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:26:59,530 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.76 MB heapSize=128.00 MB 2020-11-17 08:26:59,559 INFO [Thread-2700] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:27:00,389 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.76 MB at sequenceid=263519740 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/29b605dacdbb4b5fb26edb0bd7cd51bd 2020-11-17 08:27:00,395 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:27:00,413 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/29b605dacdbb4b5fb26edb0bd7cd51bd, entries=406843, sequenceid=263519740, filesize=8.0 M 2020-11-17 08:27:00,415 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.76 MB/103560466, heapSize ~128.00 MB/134219000, currentSize=218.46 KB/223700 for e5dc5c620be19b02666a19c280ff582e in 885ms, sequenceid=263519740, compaction requested=true 2020-11-17 08:27:00,417 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 08:27:00,417 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/37f89b84649b474c95c3ad3d56d7dbfb, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/373e8c032fd141d88f0f6825b444cd66, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/29b605dacdbb4b5fb26edb0bd7cd51bd] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=58.8 M 2020-11-17 08:27:00,441 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:27:00,444 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:27:00,469 INFO [Thread-2702] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:27:12,800 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#592 average throughput is 57.08 MB/second, slept 14 time(s) and total slept time is 4774 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:27:12,946 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into b3849078954145149628dea36060a8ee(size=60.2 M), total size for store is 49.4 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 08:27:12,946 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605601620416; duration=12sec 2020-11-17 08:27:31,599 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601537479 with entries=34300, filesize=128.39 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601651560 2020-11-17 08:27:32,379 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:28:06,828 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:28:28,245 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.82 MB heapSize=128.00 MB 2020-11-17 08:28:28,290 INFO [Thread-2708] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:28:29,026 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.82 MB at sequenceid=272311645 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/71420f163809432398dcf640692dc16b 2020-11-17 08:28:29,032 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:28:29,049 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/71420f163809432398dcf640692dc16b, entries=318069, sequenceid=272311645, filesize=8.6 M 2020-11-17 08:28:29,052 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.82 MB/109907610, heapSize ~128.00 MB/134219288, currentSize=25.67 KB/26281 for 0469ce862509756add4a8cdb00bc465c in 808ms, sequenceid=272311645, compaction requested=true 2020-11-17 08:28:29,053 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 08:28:29,053 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/8a38ba30c01a4efc9d9d33f502eba9b5, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/74d2dbf7b63a49f3a186fa37fed61574, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/71420f163809432398dcf640692dc16b] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=26.0 M 2020-11-17 08:28:29,089 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:28:29,092 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:28:29,113 INFO [Thread-2710] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:28:34,420 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#594 average throughput is 59.06 MB/second, slept 6 time(s) and total slept time is 2383 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:28:34,467 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into a14bacb4783244a7824394bbd6c46147(size=26.9 M), total size for store is 42.8 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 08:28:34,467 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605601709052; duration=5sec 2020-11-17 08:28:37,650 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601651560 with entries=19874, filesize=128.19 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601717628 2020-11-17 08:28:53,576 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.99 MB heapSize=128.41 MB 2020-11-17 08:28:53,613 INFO [Thread-2712] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:28:54,279 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.99 MB at sequenceid=273140357 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/76acc5d5cb5e468f8999a4e359b48506 2020-11-17 08:28:54,287 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:28:54,307 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/76acc5d5cb5e468f8999a4e359b48506, entries=278967, sequenceid=273140357, filesize=8.4 M 2020-11-17 08:28:54,309 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~107.99 MB/113240522, heapSize ~128.41 MB/134649192, currentSize=727.69 KB/745155 for fc93f6dcc564341dcc983bf30971f73d in 733ms, sequenceid=273140357, compaction requested=true 2020-11-17 08:28:54,310 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 08:28:54,310 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/30625b3021ca484986c7a2bc29e77ce5, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/5804e2f879184cf9bbf986cf9a3fbb7f, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/76acc5d5cb5e468f8999a4e359b48506] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=112.8 M 2020-11-17 08:28:54,346 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:28:54,351 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:28:54,376 INFO [Thread-2714] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:28:55,138 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:28:59,623 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 24351 ns. 2020-11-17 08:28:59,623 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 23347 ns. 2020-11-17 08:29:12,803 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:29:12,805 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:29:13,755 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:29:16,264 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:29:18,157 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#596 average throughput is 58.50 MB/second, slept 26 time(s) and total slept time is 12291 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:29:18,239 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into 6615e712faea4e89a6ca25151bb6490e(size=112.8 M), total size for store is 33.4 G. This selection was in queue for 0sec, and took 23sec to execute. 2020-11-17 08:29:18,239 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=11, startTime=1605601734309; duration=23sec 2020-11-17 08:29:24,854 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.68 MB, max=1.83 GB, blockCount=44209, accesses=25533842, hits=21799134, hitRatio=85.37%, , cachingAccesses=21850933, cachingHits=21794282, cachingHitsRatio=99.74%, evictions=3239, evicted=0, evictedPerRun=0.0 2020-11-17 08:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 08:29:38,779 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 370179 ns. 2020-11-17 08:29:38,780 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 68996 ns. 2020-11-17 08:29:38,780 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 58503 ns. 2020-11-17 08:29:38,780 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 193454 ns. 2020-11-17 08:29:38,780 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 48292 ns. 2020-11-17 08:29:49,010 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.38 MB heapSize=128.22 MB 2020-11-17 08:29:49,032 INFO [Thread-2719] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:29:49,688 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.38 MB at sequenceid=252149196 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/7cd7855e5c48490fb53bf5c759ff424d 2020-11-17 08:29:49,694 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:29:49,713 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/7cd7855e5c48490fb53bf5c759ff424d, entries=312058, sequenceid=252149196, filesize=6.4 M 2020-11-17 08:29:49,717 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.38 MB/111552286, heapSize ~128.22 MB/134445520, currentSize=43.79 KB/44841 for 82421ce8c5c8b48c573baee83caa8305 in 707ms, sequenceid=252149196, compaction requested=true 2020-11-17 08:29:49,718 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 08:29:49,718 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b5035d1014a849bd88807e0eca4109d6, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/2ae3ae70955a4f70bddc8f02fc6b0cbb, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/7cd7855e5c48490fb53bf5c759ff424d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=83.1 M 2020-11-17 08:29:49,748 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:29:49,750 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:29:49,782 INFO [Thread-2721] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:30:11,125 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:30:11,128 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:30:13,733 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#598 average throughput is 57.39 MB/second, slept 25 time(s) and total slept time is 11632 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:30:13,823 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 89ccc9a3d3524f6c82fe28a7498ae201(size=82.9 M), total size for store is 11.3 G. This selection was in queue for 0sec, and took 24sec to execute. 2020-11-17 08:30:13,823 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605601789717; duration=24sec 2020-11-17 08:30:30,401 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601717628 with entries=34345, filesize=128.30 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601830371 2020-11-17 08:30:30,408 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.82 MB heapSize=128.51 MB 2020-11-17 08:30:30,436 INFO [Thread-2725] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:30:31,196 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.82 MB at sequenceid=263528599 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/d1dbd14846c5450697a52fb1e4d33971 2020-11-17 08:30:31,202 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:30:31,216 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d1dbd14846c5450697a52fb1e4d33971, entries=384427, sequenceid=263528599, filesize=8.0 M 2020-11-17 08:30:31,218 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.82 MB/105717609, heapSize ~128.51 MB/134753312, currentSize=43.28 KB/44323 for e5dc5c620be19b02666a19c280ff582e in 810ms, sequenceid=263528599, compaction requested=true 2020-11-17 08:30:40,762 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.28 MB heapSize=128.02 MB 2020-11-17 08:30:40,798 INFO [Thread-2727] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:30:41,853 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.28 MB at sequenceid=262057116 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/06e112e340f546448b70252ee837fdf8 2020-11-17 08:30:41,859 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:30:41,874 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/06e112e340f546448b70252ee837fdf8, entries=582841, sequenceid=262057116, filesize=8.8 M 2020-11-17 08:30:41,876 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.28 MB/89427192, heapSize ~128.02 MB/134236984, currentSize=104.27 KB/106775 for 73aae5715542e1a5370f7540d2117abc in 1114ms, sequenceid=262057116, compaction requested=true 2020-11-17 08:30:41,877 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 08:30:41,877 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/2ab60e7e1f44419e99ce73547f03b85a, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/54b2328cc63040a39a57bf4962cf48b5, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/06e112e340f546448b70252ee837fdf8] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=81.4 M 2020-11-17 08:30:41,918 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:30:41,922 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:30:41,946 INFO [Thread-2729] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:30:48,568 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:30:52,158 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:30:54,980 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#601 average throughput is 57.31 MB/second, slept 11 time(s) and total slept time is 2284 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:30:55,054 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 56ffd78326844f6bae4b1973f421a7d1(size=81.9 M), total size for store is 12.6 G. This selection was in queue for 0sec, and took 13sec to execute. 2020-11-17 08:30:55,054 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=10, startTime=1605601841877; duration=13sec 2020-11-17 08:31:11,499 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:31:17,817 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:31:38,070 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:31:49,900 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:31:53,675 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:31:59,475 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601830371 with entries=27231, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601919438 2020-11-17 08:32:04,648 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:32:08,920 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:32:15,704 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:32:32,210 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:32:38,059 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:32:41,203 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:32:47,394 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:32:58,772 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:07,083 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:15,986 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.09 MB heapSize=128.00 MB 2020-11-17 08:33:16,060 INFO [Thread-2740] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:17,248 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.09 MB at sequenceid=291760162 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/61dfc85c1d9b44758eb957f7465797b9 2020-11-17 08:33:17,272 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/61dfc85c1d9b44758eb957f7465797b9, entries=552876, sequenceid=291760162, filesize=8.7 M 2020-11-17 08:33:17,275 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~88.09 MB/92371834, heapSize ~128.00 MB/134219976, currentSize=61.96 KB/63446 for bdaa31a63b7f3d06d9909b7560333e79 in 1289ms, sequenceid=291760162, compaction requested=true 2020-11-17 08:33:17,276 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-17 08:33:17,277 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/82c0abc9c11e4b70b945d12514860a5b, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/2bc9049501824e078eda9df6f89c6f0e, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/61dfc85c1d9b44758eb957f7465797b9] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp, totalSize=61.7 M 2020-11-17 08:33:17,331 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:17,335 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:17,355 INFO [Thread-2742] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:17,567 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:21,805 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:22,823 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:23,217 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:24,403 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:24,404 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:25,168 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:25,527 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:26,288 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:26,910 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:27,830 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: bdaa31a63b7f3d06d9909b7560333e79#e#compaction#603 average throughput is 56.05 MB/second, slept 9 time(s) and total slept time is 1876 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:33:27,896 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of bdaa31a63b7f3d06d9909b7560333e79 into 343d2679fdc54cc2bd1909c1c612e0eb(size=61.2 M), total size for store is 31.9 G. This selection was in queue for 0sec, and took 10sec to execute. 2020-11-17 08:33:27,896 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., storeName=e, priority=10, startTime=1605601997275; duration=10sec 2020-11-17 08:33:38,513 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:42,623 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601919438 with entries=31648, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602022595 2020-11-17 08:33:42,624 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600138345 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605600138345 2020-11-17 08:33:42,659 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600225159 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605600225159 2020-11-17 08:33:42,668 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600326973 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605600326973 2020-11-17 08:33:42,674 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600405584 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605600405584 2020-11-17 08:33:42,706 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600525499 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605600525499 2020-11-17 08:33:42,712 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600602588 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605600602588 2020-11-17 08:33:42,722 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600714554 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605600714554 2020-11-17 08:33:42,729 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600786152 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605600786152 2020-11-17 08:33:42,741 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600900093 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605600900093 2020-11-17 08:33:46,085 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:33:59,752 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:34:02,934 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:34:05,749 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.52 MB heapSize=128.15 MB 2020-11-17 08:34:05,807 INFO [Thread-2747] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:34:06,694 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.52 MB at sequenceid=263537621 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/d1121bbb2a9345c8a3e1a3666321b0ae 2020-11-17 08:34:06,716 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d1121bbb2a9345c8a3e1a3666321b0ae, entries=355414, sequenceid=263537621, filesize=8.1 M 2020-11-17 08:34:06,726 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.52 MB/107499708, heapSize ~128.15 MB/134373552, currentSize=1.12 MB/1178430 for e5dc5c620be19b02666a19c280ff582e in 977ms, sequenceid=263537621, compaction requested=true 2020-11-17 08:34:06,730 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 08:34:06,730 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b3849078954145149628dea36060a8ee, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d1dbd14846c5450697a52fb1e4d33971, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d1121bbb2a9345c8a3e1a3666321b0ae] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=76.3 M 2020-11-17 08:34:06,770 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:34:06,772 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:34:06,805 INFO [Thread-2749] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:34:06,832 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:34:09,196 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 39003 ns. 2020-11-17 08:34:09,196 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 12303 ns. 2020-11-17 08:34:13,575 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:34:22,633 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#605 average throughput is 57.29 MB/second, slept 17 time(s) and total slept time is 4934 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:34:22,786 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into b5b86476dd864560958a4ba9882fe839(size=77.9 M), total size for store is 49.4 G. This selection was in queue for 0sec, and took 16sec to execute. 2020-11-17 08:34:22,786 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605602046729; duration=16sec 2020-11-17 08:34:30,380 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:34:35,370 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=170.14 MB, max=1.83 GB, blockCount=44239, accesses=26362426, hits=22568608, hitRatio=85.61%, , cachingAccesses=22620444, cachingHits=22563679, cachingHitsRatio=99.75%, evictions=3269, evicted=0, evictedPerRun=0.0 2020-11-17 08:34:39,310 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 56281 ns. 2020-11-17 08:34:39,310 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 59179 ns. 2020-11-17 08:35:12,404 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602022595 with entries=27088, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602112359 2020-11-17 08:35:25,791 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:35:35,990 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.01 MB heapSize=128.07 MB 2020-11-17 08:35:36,036 INFO [Thread-2754] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:35:36,826 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.01 MB at sequenceid=260229307 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/1d87344aea454e379fa0ae23b2c540a5 2020-11-17 08:35:36,834 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:35:36,867 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/1d87344aea454e379fa0ae23b2c540a5, entries=378374, sequenceid=260229307, filesize=7.1 M 2020-11-17 08:35:36,869 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.01 MB/105920308, heapSize ~128.07 MB/134289544, currentSize=112.86 KB/115570 for 06f7ca9534710e37d90d9f8345864259 in 880ms, sequenceid=260229307, compaction requested=true 2020-11-17 08:35:36,870 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 08:35:36,871 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/90a7ee8828c34b04addff0d4aa3c9f37, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/0236f1f9b5114fa8aa51a9f77cccd6a7, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/1d87344aea454e379fa0ae23b2c540a5] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=96.2 M 2020-11-17 08:35:36,901 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:35:36,906 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:35:36,931 INFO [Thread-2756] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:35:39,561 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 143968 ns. 2020-11-17 08:35:39,562 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 452298 ns. 2020-11-17 08:35:39,562 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 54207 ns. 2020-11-17 08:35:45,887 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:35:45,889 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:35:47,509 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:35:59,233 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#607 average throughput is 57.92 MB/second, slept 24 time(s) and total slept time is 8132 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:35:59,332 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 836ae9c20d5e4baf9b0e59086c572b37(size=96.5 M), total size for store is 55.5 G. This selection was in queue for 0sec, and took 22sec to execute. 2020-11-17 08:35:59,332 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=11, startTime=1605602136870; duration=22sec 2020-11-17 08:36:06,658 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.35 MB heapSize=128.33 MB 2020-11-17 08:36:06,698 INFO [Thread-2759] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:36:07,458 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.35 MB at sequenceid=252157386 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/faf0150918874012af83b2fa9d0ffe5c 2020-11-17 08:36:07,463 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:36:07,480 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/faf0150918874012af83b2fa9d0ffe5c, entries=313990, sequenceid=252157386, filesize=6.4 M 2020-11-17 08:36:07,481 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.35 MB/111518961, heapSize ~128.33 MB/134559048, currentSize=154.20 KB/157902 for 82421ce8c5c8b48c573baee83caa8305 in 823ms, sequenceid=252157386, compaction requested=true 2020-11-17 08:36:48,015 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:36:52,681 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602112359 with entries=26960, filesize=128.20 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602212654 2020-11-17 08:38:02,028 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.83 MB heapSize=128.01 MB 2020-11-17 08:38:02,079 INFO [Thread-2766] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:38:02,862 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.83 MB at sequenceid=263547070 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/5223f1c07f614537aac7aca2f7b161c3 2020-11-17 08:38:02,868 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:38:02,895 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5223f1c07f614537aac7aca2f7b161c3, entries=362922, sequenceid=263547070, filesize=8.0 M 2020-11-17 08:38:02,897 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.83 MB/106778808, heapSize ~128.01 MB/134226592, currentSize=43.21 KB/44246 for e5dc5c620be19b02666a19c280ff582e in 869ms, sequenceid=263547070, compaction requested=true 2020-11-17 08:38:13,203 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602212654 with entries=24707, filesize=128.09 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602293168 2020-11-17 08:39:19,449 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 38427 ns. 2020-11-17 08:39:19,449 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 24134 ns. 2020-11-17 08:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.17 MB, max=1.83 GB, blockCount=44208, accesses=26570281, hits=22755416, hitRatio=85.64%, , cachingAccesses=22807261, cachingHits=22750472, cachingHitsRatio=99.75%, evictions=3299, evicted=0, evictedPerRun=0.0 2020-11-17 08:39:40,777 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602293168 with entries=27236, filesize=128.09 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602380736 2020-11-17 08:39:52,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:40:39,040 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 79591 ns. 2020-11-17 08:40:39,041 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 43235 ns. 2020-11-17 08:41:09,173 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.88 MB heapSize=128.75 MB 2020-11-17 08:41:09,205 INFO [Thread-2775] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:41:09,780 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.88 MB at sequenceid=252163487 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/af561b62bfb444518d58460b5ab85a86 2020-11-17 08:41:09,788 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:41:09,813 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/af561b62bfb444518d58460b5ab85a86, entries=312673, sequenceid=252163487, filesize=6.3 M 2020-11-17 08:41:09,816 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.88 MB/112076034, heapSize ~128.75 MB/135008384, currentSize=45.29 KB/46376 for 82421ce8c5c8b48c573baee83caa8305 in 643ms, sequenceid=252163487, compaction requested=true 2020-11-17 08:41:09,817 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 08:41:09,817 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/89ccc9a3d3524f6c82fe28a7498ae201, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/faf0150918874012af83b2fa9d0ffe5c, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/af561b62bfb444518d58460b5ab85a86] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=95.7 M 2020-11-17 08:41:09,843 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:41:09,846 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:41:09,869 INFO [Thread-2777] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:41:12,198 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602380736 with entries=27587, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602472174 2020-11-17 08:41:34,112 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.28 MB heapSize=128.00 MB 2020-11-17 08:41:34,166 INFO [Thread-2780] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:41:35,104 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.28 MB at sequenceid=287930905 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/c3b2e9147ae147928d0d5aeceed608c0 2020-11-17 08:41:35,110 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:41:35,125 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/c3b2e9147ae147928d0d5aeceed608c0, entries=526943, sequenceid=287930905, filesize=8.9 M 2020-11-17 08:41:35,127 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.28 MB/94669213, heapSize ~128.00 MB/134219200, currentSize=40.13 KB/41093 for 5e5f05205d7e40b0141cd977dd009ba1 in 1015ms, sequenceid=287930905, compaction requested=true 2020-11-17 08:41:35,404 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:41:38,254 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#611 average throughput is 55.80 MB/second, slept 28 time(s) and total slept time is 11880 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:41:38,329 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 5b086873b1204452b7ddd83edef3d5cf(size=95.3 M), total size for store is 11.3 G. This selection was in queue for 0sec, and took 28sec to execute. 2020-11-17 08:41:38,329 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605602469816; duration=28sec 2020-11-17 08:41:39,590 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 62309 ns. 2020-11-17 08:41:39,590 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 164126 ns. 2020-11-17 08:41:39,591 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 478979 ns. 2020-11-17 08:42:02,597 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.64 MB heapSize=128.00 MB 2020-11-17 08:42:02,634 INFO [Thread-2784] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:42:03,362 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.64 MB at sequenceid=272345239 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/3469d3aafbb54c4abeb77cd463af9a79 2020-11-17 08:42:03,368 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:42:03,392 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/3469d3aafbb54c4abeb77cd463af9a79, entries=321147, sequenceid=272345239, filesize=8.6 M 2020-11-17 08:42:03,394 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.64 MB/109724560, heapSize ~128.00 MB/134217816, currentSize=32.31 KB/33084 for 0469ce862509756add4a8cdb00bc465c in 797ms, sequenceid=272345239, compaction requested=true 2020-11-17 08:42:14,400 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:42:27,190 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:42:34,817 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=97.61 MB heapSize=128.01 MB 2020-11-17 08:42:34,851 INFO [Thread-2788] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:42:35,640 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.61 MB at sequenceid=263558467 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/6df47d0df6d2451dae15b0faf8c4b7fb 2020-11-17 08:42:35,647 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:42:35,663 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6df47d0df6d2451dae15b0faf8c4b7fb, entries=423258, sequenceid=263558467, filesize=7.8 M 2020-11-17 08:42:35,665 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~97.61 MB/102348673, heapSize ~128.01 MB/134232848, currentSize=1.25 MB/1307997 for e5dc5c620be19b02666a19c280ff582e in 847ms, sequenceid=263558467, compaction requested=true 2020-11-17 08:42:35,665 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 08:42:35,665 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b5b86476dd864560958a4ba9882fe839, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5223f1c07f614537aac7aca2f7b161c3, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6df47d0df6d2451dae15b0faf8c4b7fb] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=93.7 M 2020-11-17 08:42:35,705 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:42:35,707 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:42:35,732 INFO [Thread-2790] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:42:51,242 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602472174 with entries=29587, filesize=128.37 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602571193 2020-11-17 08:42:51,242 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605600975468 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605600975468 2020-11-17 08:42:51,259 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601082666 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605601082666 2020-11-17 08:42:51,268 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601168707 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605601168707 2020-11-17 08:42:51,275 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601276190 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605601276190 2020-11-17 08:42:51,280 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601352723 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605601352723 2020-11-17 08:42:51,286 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601453613 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605601453613 2020-11-17 08:42:51,291 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601537479 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605601537479 2020-11-17 08:42:51,300 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601651560 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605601651560 2020-11-17 08:42:54,988 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#615 average throughput is 57.47 MB/second, slept 21 time(s) and total slept time is 5937 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 08:42:55,049 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 7fd8ce2b2bab4573af7b860dd2505239(size=95.3 M), total size for store is 49.5 G. This selection was in queue for 0sec, and took 19sec to execute. 2020-11-17 08:42:55,050 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605602555665; duration=19sec 2020-11-17 08:43:02,075 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:43:18,065 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:44:08,170 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.01 MB heapSize=128.23 MB 2020-11-17 08:44:08,204 INFO [Thread-2795] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:44:08,893 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.01 MB at sequenceid=273159794 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/c22222f4220c48be9c29f4d89aadee60 2020-11-17 08:44:08,899 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:44:08,917 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/c22222f4220c48be9c29f4d89aadee60, entries=277459, sequenceid=273159794, filesize=8.5 M 2020-11-17 08:44:08,928 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.01 MB/113259554, heapSize ~128.23 MB/134462712, currentSize=10.36 KB/10609 for fc93f6dcc564341dcc983bf30971f73d in 758ms, sequenceid=273159794, compaction requested=true 2020-11-17 08:44:09,697 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602571193 with entries=23486, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602649666 2020-11-17 08:44:09,698 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601717628 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605601717628 2020-11-17 08:44:29,466 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 31125 ns. 2020-11-17 08:44:29,466 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 19366 ns. 2020-11-17 08:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.94 MB, max=1.83 GB, blockCount=44196, accesses=26670586, hits=22812040, hitRatio=85.53%, , cachingAccesses=22863877, cachingHits=22807081, cachingHitsRatio=99.75%, evictions=3329, evicted=0, evictedPerRun=0.0 2020-11-17 08:45:12,956 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:45:24,128 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:45:26,389 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.74 MB heapSize=128.00 MB 2020-11-17 08:45:26,433 INFO [Thread-2800] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:45:27,130 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.74 MB at sequenceid=260254805 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/b48ac45a030b40d79b9b9381c6ec1823 2020-11-17 08:45:27,136 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:45:27,165 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/b48ac45a030b40d79b9b9381c6ec1823, entries=377617, sequenceid=260254805, filesize=6.9 M 2020-11-17 08:45:27,167 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.74 MB/105636989, heapSize ~128.00 MB/134217928, currentSize=38.22 KB/39133 for 06f7ca9534710e37d90d9f8345864259 in 777ms, sequenceid=260254805, compaction requested=true 2020-11-17 08:45:39,657 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602649666 with entries=28883, filesize=128.21 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602739629 2020-11-17 08:45:59,070 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.02 MB heapSize=128.45 MB 2020-11-17 08:45:59,103 INFO [Thread-2804] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:45:59,842 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.02 MB at sequenceid=263567171 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/bf27d42d44b04f098e88059329685f26 2020-11-17 08:45:59,847 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:45:59,865 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/bf27d42d44b04f098e88059329685f26, entries=366525, sequenceid=263567171, filesize=8.0 M 2020-11-17 08:45:59,867 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.02 MB/106979160, heapSize ~128.45 MB/134687872, currentSize=1022.27 KB/1046805 for e5dc5c620be19b02666a19c280ff582e in 797ms, sequenceid=263567171, compaction requested=true 2020-11-17 08:46:05,944 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:46:21,261 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:46:23,696 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 501 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 08:46:23,790 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=86.43 MB heapSize=128.12 MB 2020-11-17 08:46:23,819 INFO [Thread-2807] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:46:24,986 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=86.43 MB at sequenceid=262098600 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/643eec1fd4704691ac92d6d5b0caf897 2020-11-17 08:46:25,007 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/643eec1fd4704691ac92d6d5b0caf897, entries=575531, sequenceid=262098600, filesize=8.8 M 2020-11-17 08:46:25,009 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~86.43 MB/90624106, heapSize ~128.12 MB/134340480, currentSize=160.90 KB/164763 for 73aae5715542e1a5370f7540d2117abc in 1219ms, sequenceid=262098600, compaction requested=true 2020-11-17 08:46:29,946 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:46:36,293 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:46:36,893 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.19 MB heapSize=128.14 MB 2020-11-17 08:46:36,928 INFO [Thread-2810] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:46:37,543 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.20 MB at sequenceid=252170584 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/b37bc44c324943c98a105c1bd310b7e6 2020-11-17 08:46:37,563 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b37bc44c324943c98a105c1bd310b7e6, entries=328342, sequenceid=252170584, filesize=6.3 M 2020-11-17 08:46:37,565 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.20 MB/110305183, heapSize ~128.14 MB/134364240, currentSize=33.72 KB/34530 for 82421ce8c5c8b48c573baee83caa8305 in 673ms, sequenceid=252170584, compaction requested=true 2020-11-17 08:46:37,566 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 08:46:37,566 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/125dc01d08574882823d5bdf8719b7c1, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/5b086873b1204452b7ddd83edef3d5cf, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b37bc44c324943c98a105c1bd310b7e6] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=223.0 M 2020-11-17 08:46:37,612 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:46:37,615 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:46:37,638 INFO [Thread-2812] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:46:38,760 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 40620 ns. 2020-11-17 08:46:38,760 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 60527 ns. 2020-11-17 08:46:45,156 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:06,454 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:10,722 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:12,383 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602739629 with entries=29197, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602832351 2020-11-17 08:47:12,383 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601830371 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605601830371 2020-11-17 08:47:15,477 INFO [DataStreamer for file /hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/0554c2a6f7434d01b0efd803728fb16d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:17,032 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:28,133 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:35,861 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:35,867 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:35,873 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:35,877 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:36,019 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:37,305 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:37,307 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:38,792 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 78922 ns. 2020-11-17 08:47:38,792 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 217814 ns. 2020-11-17 08:47:38,793 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 383737 ns. 2020-11-17 08:47:39,953 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:39,955 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:40,224 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:40,227 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:40,877 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:47:41,807 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#621 average throughput is 57.07 MB/second, slept 70 time(s) and total slept time is 31481 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:47:41,890 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 0554c2a6f7434d01b0efd803728fb16d(size=220.9 M), total size for store is 11.3 G. This selection was in queue for 0sec, and took 1mins, 4sec to execute. 2020-11-17 08:47:41,890 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605602797565; duration=1mins, 4sec 2020-11-17 08:48:28,699 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:48:34,630 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:48:41,773 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:48:50,952 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:49:04,564 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602832351 with entries=34362, filesize=128.22 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602944515 2020-11-17 08:49:30,981 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.38 MB heapSize=128.42 MB 2020-11-17 08:49:31,054 INFO [Thread-2823] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:49:31,851 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.38 MB at sequenceid=263576191 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a6292e10339b40f4b5773d4c92cba316 2020-11-17 08:49:31,862 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:49:31,898 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a6292e10339b40f4b5773d4c92cba316, entries=361131, sequenceid=263576191, filesize=7.8 M 2020-11-17 08:49:31,903 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.38 MB/107348523, heapSize ~128.42 MB/134654032, currentSize=401.90 KB/411545 for e5dc5c620be19b02666a19c280ff582e in 922ms, sequenceid=263576191, compaction requested=true 2020-11-17 08:49:31,904 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 08:49:31,904 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7fd8ce2b2bab4573af7b860dd2505239, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/bf27d42d44b04f098e88059329685f26, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a6292e10339b40f4b5773d4c92cba316] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=111.2 M 2020-11-17 08:49:31,968 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:49:31,971 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:49:31,994 INFO [Thread-2825] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=169.89 MB, max=1.83 GB, blockCount=44237, accesses=26884880, hits=22963125, hitRatio=85.41%, , cachingAccesses=23014993, cachingHits=22958149, cachingHitsRatio=99.75%, evictions=3359, evicted=0, evictedPerRun=0.0 2020-11-17 08:49:39,623 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14257 ns. 2020-11-17 08:49:39,623 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 37787 ns. 2020-11-17 08:49:53,956 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#623 average throughput is 59.70 MB/second, slept 24 time(s) and total slept time is 6402 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 08:49:54,046 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 7db8e12ac3b24f94b98fbf9254534c7d(size=113.0 M), total size for store is 49.5 G. This selection was in queue for 0sec, and took 22sec to execute. 2020-11-17 08:49:54,047 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605602971903; duration=22sec 2020-11-17 08:50:18,685 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602944515 with entries=22858, filesize=128.41 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603018644 2020-11-17 08:50:39,860 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:50:50,506 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:50:53,999 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:51:48,072 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603018644 with entries=26172, filesize=128.34 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603108039 2020-11-17 08:51:48,405 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.41 MB heapSize=128.41 MB 2020-11-17 08:51:48,458 INFO [Thread-2832] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:51:49,180 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.41 MB at sequenceid=252177360 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/694237a18629441782fb6457d339784d 2020-11-17 08:51:49,189 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:51:49,209 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/694237a18629441782fb6457d339784d, entries=314312, sequenceid=252177360, filesize=6.4 M 2020-11-17 08:51:49,211 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.41 MB/111574049, heapSize ~128.41 MB/134643600, currentSize=489.86 KB/501616 for 82421ce8c5c8b48c573baee83caa8305 in 806ms, sequenceid=252177360, compaction requested=true 2020-11-17 08:52:38,665 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 64056 ns. 2020-11-17 08:52:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 61876 ns. 2020-11-17 08:52:53,176 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603108039 with entries=17223, filesize=128.30 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603173140 2020-11-17 08:53:39,237 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.67 MB heapSize=128.18 MB 2020-11-17 08:53:39,282 INFO [Thread-2838] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:53:39,509 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 168638 ns. 2020-11-17 08:53:39,510 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 385839 ns. 2020-11-17 08:53:39,510 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 71313 ns. 2020-11-17 08:53:40,054 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.67 MB at sequenceid=263585973 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/d475b8ebb44a4cb9a5d7a4d70e27689f 2020-11-17 08:53:40,060 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:53:40,078 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d475b8ebb44a4cb9a5d7a4d70e27689f, entries=381916, sequenceid=263585973, filesize=7.8 M 2020-11-17 08:53:40,080 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.67 MB/105557520, heapSize ~128.18 MB/134411144, currentSize=25.29 KB/25894 for e5dc5c620be19b02666a19c280ff582e in 843ms, sequenceid=263585973, compaction requested=true 2020-11-17 08:54:03,408 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:54:17,766 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603173140 with entries=24254, filesize=128.72 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603257728 2020-11-17 08:54:36,025 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.13 MB, max=1.83 GB, blockCount=44215, accesses=27033869, hits=23094606, hitRatio=85.43%, , cachingAccesses=23146480, cachingHits=23089629, cachingHitsRatio=99.75%, evictions=3389, evicted=0, evictedPerRun=0.0 2020-11-17 08:54:49,634 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 33613 ns. 2020-11-17 08:54:49,634 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15008 ns. 2020-11-17 08:55:11,402 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.75 MB heapSize=128.10 MB 2020-11-17 08:55:11,446 INFO [Thread-2843] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:12,241 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.75 MB at sequenceid=272378000 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/bd57a6bacc324f228bcf55425421c249 2020-11-17 08:55:12,246 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:12,265 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/bd57a6bacc324f228bcf55425421c249, entries=320770, sequenceid=272378000, filesize=8.6 M 2020-11-17 08:55:12,267 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.75 MB/109839791, heapSize ~128.10 MB/134324248, currentSize=54.33 KB/55638 for 0469ce862509756add4a8cdb00bc465c in 865ms, sequenceid=272378000, compaction requested=true 2020-11-17 08:55:12,268 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 08:55:12,268 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/a14bacb4783244a7824394bbd6c46147, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/3469d3aafbb54c4abeb77cd463af9a79, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/bd57a6bacc324f228bcf55425421c249] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=44.0 M 2020-11-17 08:55:12,300 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:12,304 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:12,323 INFO [Thread-2845] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:12,442 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.24 MB heapSize=128.06 MB 2020-11-17 08:55:12,473 INFO [Thread-2847] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:13,306 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.24 MB at sequenceid=260280068 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/8fa20288e69d4755b40849e32e3eb05c 2020-11-17 08:55:13,314 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:13,338 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/8fa20288e69d4755b40849e32e3eb05c, entries=374004, sequenceid=260280068, filesize=6.9 M 2020-11-17 08:55:13,339 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.24 MB/106154864, heapSize ~128.06 MB/134276456, currentSize=259.84 KB/266076 for 06f7ca9534710e37d90d9f8345864259 in 897ms, sequenceid=260280068, compaction requested=true 2020-11-17 08:55:13,340 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 08:55:13,340 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/836ae9c20d5e4baf9b0e59086c572b37, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/b48ac45a030b40d79b9b9381c6ec1823, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/8fa20288e69d4755b40849e32e3eb05c] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=110.3 M 2020-11-17 08:55:13,478 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:13,491 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:13,527 INFO [Thread-2849] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:15,475 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.47 MB heapSize=128.46 MB 2020-11-17 08:55:15,585 INFO [Thread-2851] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:16,177 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.47 MB at sequenceid=252181349 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/cd3636bb036142118043b23354c2e7ff 2020-11-17 08:55:16,182 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:16,198 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/cd3636bb036142118043b23354c2e7ff, entries=314639, sequenceid=252181349, filesize=6.2 M 2020-11-17 08:55:16,199 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.47 MB/111641660, heapSize ~128.46 MB/134699064, currentSize=920.88 KB/942980 for 82421ce8c5c8b48c573baee83caa8305 in 724ms, sequenceid=252181349, compaction requested=true 2020-11-17 08:55:24,162 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603257728 with entries=20188, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603324139 2020-11-17 08:55:28,950 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#627 average throughput is 31.42 MB/second, slept 10 time(s) and total slept time is 11889 ms. 1 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:55:29,029 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 29ceb6c76f4c4cce94933b9c504fe026(size=44.8 M), total size for store is 42.9 G. This selection was in queue for 0sec, and took 16sec to execute. 2020-11-17 08:55:29,029 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605603312267; duration=16sec 2020-11-17 08:55:29,301 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:31,569 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:31,571 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:41,969 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:47,069 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#629 average throughput is 44.43 MB/second, slept 28 time(s) and total slept time is 17899 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 08:55:47,110 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:55:47,133 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 2f1ee6a8a2b546928d4c2e9bcb9a2d08(size=110.4 M), total size for store is 55.5 G. This selection was in queue for 0sec, and took 33sec to execute. 2020-11-17 08:55:47,133 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=11, startTime=1605603313339; duration=33sec 2020-11-17 08:56:37,426 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:56:48,017 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:57:01,301 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:57:22,016 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603324139 with entries=35496, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603441991 2020-11-17 08:57:28,299 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.25 MB heapSize=128.00 MB 2020-11-17 08:57:28,325 INFO [Thread-2857] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:57:29,141 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.25 MB at sequenceid=263595445 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/64bfacb36fb941bc95a2e4b04bcba327 2020-11-17 08:57:29,147 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:57:29,164 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/64bfacb36fb941bc95a2e4b04bcba327, entries=413802, sequenceid=263595445, filesize=7.9 M 2020-11-17 08:57:29,166 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~98.25 MB/103025855, heapSize ~128.00 MB/134221432, currentSize=342.60 KB/350818 for e5dc5c620be19b02666a19c280ff582e in 867ms, sequenceid=263595445, compaction requested=true 2020-11-17 08:57:38,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 57560 ns. 2020-11-17 08:57:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 38282 ns. 2020-11-17 08:57:58,409 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:58:33,780 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:58:52,863 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:59:18,844 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.07 MB heapSize=128.49 MB 2020-11-17 08:59:18,887 INFO [Thread-2862] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:59:19,594 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.07 MB at sequenceid=273178921 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/3c24025de313435fa1515d86dac84d3c 2020-11-17 08:59:19,600 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 08:59:19,617 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/3c24025de313435fa1515d86dac84d3c, entries=280391, sequenceid=273178921, filesize=8.4 M 2020-11-17 08:59:19,618 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.07 MB/113318900, heapSize ~128.49 MB/134735936, currentSize=23.14 KB/23697 for fc93f6dcc564341dcc983bf30971f73d in 774ms, sequenceid=273178921, compaction requested=true 2020-11-17 08:59:24,000 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603441991 with entries=37053, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603563978 2020-11-17 08:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.58 MB, max=1.83 GB, blockCount=44206, accesses=27080809, hits=23108810, hitRatio=85.33%, , cachingAccesses=23160692, cachingHits=23103814, cachingHitsRatio=99.75%, evictions=3419, evicted=0, evictedPerRun=0.0 2020-11-17 08:59:38,909 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 354786 ns. 2020-11-17 08:59:38,909 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 43219 ns. 2020-11-17 08:59:38,909 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 173007 ns. 2020-11-17 08:59:59,496 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 9362 ns. 2020-11-17 08:59:59,496 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 32694 ns. 2020-11-17 09:01:04,785 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.63 MB heapSize=128.17 MB 2020-11-17 09:01:04,825 INFO [Thread-2868] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:01:05,600 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.63 MB at sequenceid=263604758 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/5e22244ac067449089d1351fc11bb639 2020-11-17 09:01:05,606 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:01:05,626 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5e22244ac067449089d1351fc11bb639, entries=368495, sequenceid=263604758, filesize=8.1 M 2020-11-17 09:01:05,627 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.63 MB/106567999, heapSize ~128.17 MB/134391848, currentSize=647.30 KB/662837 for e5dc5c620be19b02666a19c280ff582e in 842ms, sequenceid=263604758, compaction requested=true 2020-11-17 09:01:05,628 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 09:01:05,629 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d475b8ebb44a4cb9a5d7a4d70e27689f, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/64bfacb36fb941bc95a2e4b04bcba327, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5e22244ac067449089d1351fc11bb639] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=23.8 M 2020-11-17 09:01:05,655 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:01:05,658 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:01:05,685 INFO [Thread-2870] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:01:10,550 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#634 average throughput is 61.54 MB/second, slept 6 time(s) and total slept time is 1544 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:01:10,627 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 569d4e6e5b8d405889a8f34c28156622(size=24.8 M), total size for store is 49.5 G. This selection was in queue for 0sec, and took 4sec to execute. 2020-11-17 09:01:10,627 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605603665627; duration=4sec 2020-11-17 09:01:19,965 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603563978 with entries=36389, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603679943 2020-11-17 09:02:02,731 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:02:07,048 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:02:25,065 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.68 MB heapSize=128.04 MB 2020-11-17 09:02:25,104 INFO [Thread-2875] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:02:26,248 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.68 MB at sequenceid=262140499 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/c5183c993621485abff1dda123ae56f6 2020-11-17 09:02:26,253 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:02:26,269 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/c5183c993621485abff1dda123ae56f6, entries=580146, sequenceid=262140499, filesize=8.7 M 2020-11-17 09:02:26,271 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.68 MB/89845902, heapSize ~128.04 MB/134256944, currentSize=82.40 KB/84376 for 73aae5715542e1a5370f7540d2117abc in 1206ms, sequenceid=262140499, compaction requested=true 2020-11-17 09:02:26,271 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 09:02:26,272 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/35c7c16016f2485cb35e87d6bd68e5f0, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/56ffd78326844f6bae4b1973f421a7d1, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/643eec1fd4704691ac92d6d5b0caf897, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/c5183c993621485abff1dda123ae56f6] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=217.9 M 2020-11-17 09:02:26,316 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:02:26,319 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:02:26,324 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:02:26,349 INFO [Thread-2877] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:02:35,562 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:02:36,359 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.22 MB heapSize=128.00 MB 2020-11-17 09:02:36,403 INFO [Thread-2880] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:02:37,523 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.22 MB at sequenceid=291837093 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/3e5914bf38b84441a8f43f5b5ee67c60 2020-11-17 09:02:37,542 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/3e5914bf38b84441a8f43f5b5ee67c60, entries=552356, sequenceid=291837093, filesize=8.5 M 2020-11-17 09:02:37,543 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~88.22 MB/92503867, heapSize ~128.00 MB/134218880, currentSize=38.20 KB/39119 for bdaa31a63b7f3d06d9909b7560333e79 in 1185ms, sequenceid=291837093, compaction requested=true 2020-11-17 09:02:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 82938 ns. 2020-11-17 09:02:45,982 INFO [DataStreamer for file /hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/4085aa36dee1496ebd755a1c78008085] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:02:53,012 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:02:53,013 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:02:53,016 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:02:54,406 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:02:56,900 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:03:00,139 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#636 average throughput is 57.84 MB/second, slept 33 time(s) and total slept time is 6434 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:03:00,196 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 4085aa36dee1496ebd755a1c78008085(size=216.7 M), total size for store is 12.6 G. This selection was in queue for 0sec, and took 33sec to execute. 2020-11-17 09:03:00,196 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=10, startTime=1605603746271; duration=33sec 2020-11-17 09:03:06,167 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:03:20,730 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:03:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 36861 ns. 2020-11-17 09:03:40,625 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603679943 with entries=42922, filesize=128.15 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603820604 2020-11-17 09:03:40,625 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605601919438 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605601919438 2020-11-17 09:03:40,634 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602022595 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605602022595 2020-11-17 09:03:40,640 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602112359 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605602112359 2020-11-17 09:03:40,646 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602212654 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605602212654 2020-11-17 09:03:40,651 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602293168 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605602293168 2020-11-17 09:03:40,656 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602380736 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605602380736 2020-11-17 09:03:42,061 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:03:55,192 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:04:21,375 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.43 MB heapSize=128.23 MB 2020-11-17 09:04:21,405 INFO [Thread-2887] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:04:22,293 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.43 MB at sequenceid=263613172 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/442d3f440dd9497dbd8552878d1199b9 2020-11-17 09:04:22,298 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:04:22,314 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/442d3f440dd9497dbd8552878d1199b9, entries=343858, sequenceid=263613172, filesize=8.1 M 2020-11-17 09:04:22,316 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~103.43 MB/108456201, heapSize ~128.23 MB/134456432, currentSize=632.55 KB/647733 for e5dc5c620be19b02666a19c280ff582e in 941ms, sequenceid=263613172, compaction requested=true 2020-11-17 09:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.07 MB, max=1.83 GB, blockCount=44158, accesses=27263416, hits=23254269, hitRatio=85.29%, , cachingAccesses=23306112, cachingHits=23249199, cachingHitsRatio=99.76%, evictions=3449, evicted=0, evictedPerRun=0.0 2020-11-17 09:04:39,053 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:04:46,586 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.35 MB heapSize=128.00 MB 2020-11-17 09:04:46,636 INFO [Thread-2890] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:04:47,636 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.35 MB at sequenceid=287994123 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/2d413e147ee64c078b2592390fb7885f 2020-11-17 09:04:47,643 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:04:47,665 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/2d413e147ee64c078b2592390fb7885f, entries=525785, sequenceid=287994123, filesize=8.9 M 2020-11-17 09:04:47,667 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.35 MB/94740556, heapSize ~128.00 MB/134219152, currentSize=24.13 KB/24713 for 5e5f05205d7e40b0141cd977dd009ba1 in 1081ms, sequenceid=287994123, compaction requested=true 2020-11-17 09:04:47,667 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 09:04:47,668 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/60ca2181edae4ee081597535d8e8d628, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/c3b2e9147ae147928d0d5aeceed608c0, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/2d413e147ee64c078b2592390fb7885f] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=45.3 M 2020-11-17 09:04:47,701 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:04:47,723 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:04:47,772 INFO [Thread-2892] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:04:53,998 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:04:54,001 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:04:55,201 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:04:55,603 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#640 average throughput is 56.89 MB/second, slept 5 time(s) and total slept time is 1009 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:04:55,691 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into d81b2885d2884606b73e454af1810d96(size=45.7 M), total size for store is 24.4 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 09:04:55,691 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=10, startTime=1605603887667; duration=8sec 2020-11-17 09:05:03,788 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:05:07,973 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:05:09,114 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 17389 ns. 2020-11-17 09:05:09,114 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 33892 ns. 2020-11-17 09:05:23,091 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:05:27,587 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:05:32,837 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:05:38,739 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 168689 ns. 2020-11-17 09:05:38,739 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 338421 ns. 2020-11-17 09:05:38,740 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 65336 ns. 2020-11-17 09:05:41,107 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:05:47,459 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:05:57,416 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603820604 with entries=41235, filesize=128.19 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603957370 2020-11-17 09:05:57,417 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602472174 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605602472174 2020-11-17 09:05:57,428 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602571193 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605602571193 2020-11-17 09:05:57,435 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602649666 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605602649666 2020-11-17 09:05:57,441 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602739629 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605602739629 2020-11-17 09:05:57,447 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602832351 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605602832351 2020-11-17 09:05:57,453 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605602944515 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605602944515 2020-11-17 09:05:57,459 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603018644 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605603018644 2020-11-17 09:05:57,464 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603108039 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605603108039 2020-11-17 09:05:57,469 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603173140 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605603173140 2020-11-17 09:06:02,292 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:06:12,599 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:06:24,403 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:06:36,304 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:07:07,659 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:07:10,720 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:07:10,721 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:07:34,443 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:07:41,735 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603957370 with entries=31542, filesize=128.46 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604061703 2020-11-17 09:07:50,653 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:07:57,963 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:08:36,324 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.76 MB heapSize=128.00 MB 2020-11-17 09:08:36,357 INFO [Thread-2904] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:08:37,066 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.76 MB at sequenceid=272411967 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/5b7556b2e9f44794baa94e2443a64d92 2020-11-17 09:08:37,072 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:08:37,088 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/5b7556b2e9f44794baa94e2443a64d92, entries=319531, sequenceid=272411967, filesize=8.6 M 2020-11-17 09:08:37,090 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.76 MB/109849218, heapSize ~128.00 MB/134219464, currentSize=643.75 KB/659199 for 0469ce862509756add4a8cdb00bc465c in 767ms, sequenceid=272411967, compaction requested=true 2020-11-17 09:08:39,308 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 59891 ns. 2020-11-17 09:08:39,309 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 35282 ns. 2020-11-17 09:08:54,975 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:09:16,551 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.56 MB heapSize=128.00 MB 2020-11-17 09:09:16,590 INFO [Thread-2908] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:09:17,372 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.56 MB at sequenceid=263625333 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/1ca0130d4d1e485dbd0013de22b25dfe 2020-11-17 09:09:17,378 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:09:17,395 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1ca0130d4d1e485dbd0013de22b25dfe, entries=396305, sequenceid=263625333, filesize=8.0 M 2020-11-17 09:09:17,397 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~99.56 MB/104394534, heapSize ~128.00 MB/134220560, currentSize=143.13 KB/146563 for e5dc5c620be19b02666a19c280ff582e in 846ms, sequenceid=263625333, compaction requested=true 2020-11-17 09:09:17,398 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 09:09:17,398 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/569d4e6e5b8d405889a8f34c28156622, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/442d3f440dd9497dbd8552878d1199b9, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1ca0130d4d1e485dbd0013de22b25dfe] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=40.9 M 2020-11-17 09:09:17,424 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:09:17,426 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:09:17,453 INFO [Thread-2910] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:09:25,610 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#643 average throughput is 61.59 MB/second, slept 10 time(s) and total slept time is 2731 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:09:25,682 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 7cc0b1c8ff9541ab806c50b833f3f91b(size=42.4 M), total size for store is 49.5 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 09:09:25,682 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605604157397; duration=8sec 2020-11-17 09:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=170.15 MB, max=1.83 GB, blockCount=44237, accesses=28108597, hits=24083852, hitRatio=85.68%, , cachingAccesses=24135761, cachingHits=24078745, cachingHitsRatio=99.76%, evictions=3479, evicted=0, evictedPerRun=0.0 2020-11-17 09:10:09,480 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604061703 with entries=44290, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604209438 2020-11-17 09:10:18,900 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 8332 ns. 2020-11-17 09:10:18,900 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 29402 ns. 2020-11-17 09:10:18,948 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:11:38,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 190476 ns. 2020-11-17 09:11:38,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 335457 ns. 2020-11-17 09:11:38,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 57256 ns. 2020-11-17 09:12:27,647 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604209438 with entries=42441, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604347614 2020-11-17 09:13:01,484 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.60 MB heapSize=128.24 MB 2020-11-17 09:13:01,535 INFO [Thread-2921] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:13:02,429 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.60 MB at sequenceid=263634657 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/e0aee09ab74b445c934c3a68ec8da6a8 2020-11-17 09:13:02,436 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:13:02,452 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e0aee09ab74b445c934c3a68ec8da6a8, entries=384144, sequenceid=263634657, filesize=8.1 M 2020-11-17 09:13:02,454 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.60 MB/105490315, heapSize ~128.24 MB/134473064, currentSize=639.14 KB/654475 for e5dc5c620be19b02666a19c280ff582e in 970ms, sequenceid=263634657, compaction requested=true 2020-11-17 09:13:06,044 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:13:17,761 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:13:21,072 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:13:31,943 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:13:39,652 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 64092 ns. 2020-11-17 09:13:39,652 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 400234 ns. 2020-11-17 09:13:41,025 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:14:10,399 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:14:15,091 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:14:29,936 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604347614 with entries=35907, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604469887 2020-11-17 09:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=170.69 MB, max=1.83 GB, blockCount=44229, accesses=28119188, hits=24094429, hitRatio=85.69%, , cachingAccesses=24146352, cachingHits=24089322, cachingHitsRatio=99.76%, evictions=3509, evicted=0, evictedPerRun=0.0 2020-11-17 09:14:58,773 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.10 MB heapSize=128.10 MB 2020-11-17 09:14:58,812 INFO [Thread-2929] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:14:59,463 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.10 MB at sequenceid=273199100 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/f69eb6a483384ee7af14a1282537dbc2 2020-11-17 09:14:59,469 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:14:59,487 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/f69eb6a483384ee7af14a1282537dbc2, entries=274163, sequenceid=273199100, filesize=8.5 M 2020-11-17 09:14:59,489 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.10 MB/113346695, heapSize ~128.10 MB/134323816, currentSize=640.52 KB/655888 for fc93f6dcc564341dcc983bf30971f73d in 716ms, sequenceid=273199100, compaction requested=true 2020-11-17 09:14:59,490 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 09:14:59,491 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/c22222f4220c48be9c29f4d89aadee60, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/3c24025de313435fa1515d86dac84d3c, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/f69eb6a483384ee7af14a1282537dbc2] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=25.4 M 2020-11-17 09:14:59,520 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:14:59,529 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:14:59,551 INFO [Thread-2931] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:15:04,621 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#646 average throughput is 63.71 MB/second, slept 6 time(s) and total slept time is 2408 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:15:04,682 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into ef3eb15755c94e69a80dcfa9f016e93b(size=26.0 M), total size for store is 33.5 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 09:15:04,682 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605604499489; duration=5sec 2020-11-17 09:15:12,525 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:15:19,005 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 13092 ns. 2020-11-17 09:15:19,005 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 25854 ns. 2020-11-17 09:16:10,146 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:16:23,904 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604469887 with entries=35491, filesize=128.34 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604583861 2020-11-17 09:16:24,739 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=104.04 MB heapSize=128.43 MB 2020-11-17 09:16:24,771 INFO [Thread-2936] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:16:25,478 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.04 MB at sequenceid=263643338 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/466f32f087654e86a8f4f9498d5c87bf 2020-11-17 09:16:25,485 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:16:25,502 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/466f32f087654e86a8f4f9498d5c87bf, entries=337918, sequenceid=263643338, filesize=8.0 M 2020-11-17 09:16:25,504 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.04 MB/109098715, heapSize ~128.43 MB/134665736, currentSize=391.51 KB/400911 for e5dc5c620be19b02666a19c280ff582e in 765ms, sequenceid=263643338, compaction requested=true 2020-11-17 09:16:25,505 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 09:16:25,505 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7cc0b1c8ff9541ab806c50b833f3f91b, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e0aee09ab74b445c934c3a68ec8da6a8, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/466f32f087654e86a8f4f9498d5c87bf] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=58.5 M 2020-11-17 09:16:25,534 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:16:25,536 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:16:25,559 INFO [Thread-2938] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:16:37,090 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#648 average throughput is 61.32 MB/second, slept 14 time(s) and total slept time is 4264 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:16:37,166 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into c9b6047e152a4eaa80a25470c0ba0b22(size=60.0 M), total size for store is 49.5 G. This selection was in queue for 0sec, and took 11sec to execute. 2020-11-17 09:16:37,166 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605604585504; duration=11sec 2020-11-17 09:16:48,007 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:17:39,239 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 218732 ns. 2020-11-17 09:17:39,239 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 66983 ns. 2020-11-17 09:17:39,240 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 364719 ns. 2020-11-17 09:18:02,608 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:18:04,689 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=96.32 MB heapSize=128.00 MB 2020-11-17 09:18:04,731 INFO [Thread-2946] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:18:05,611 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=96.32 MB at sequenceid=260339848 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/524669ec5593465f85903a5ab063e843 2020-11-17 09:18:05,617 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:18:05,634 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/524669ec5593465f85903a5ab063e843, entries=430885, sequenceid=260339848, filesize=8.2 M 2020-11-17 09:18:05,636 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~96.32 MB/101003414, heapSize ~128.00 MB/134217824, currentSize=120.90 KB/123800 for 06f7ca9534710e37d90d9f8345864259 in 947ms, sequenceid=260339848, compaction requested=true 2020-11-17 09:18:17,522 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:18:39,275 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604583861 with entries=40716, filesize=128.51 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604719189 2020-11-17 09:18:53,615 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:19:17,998 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:19:21,655 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.29 MB heapSize=128.06 MB 2020-11-17 09:19:21,700 INFO [Thread-2951] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:19:22,987 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.29 MB at sequenceid=262185162 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/87c95deb14e444839b10c886c1cced47 2020-11-17 09:19:22,995 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:19:23,015 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/87c95deb14e444839b10c886c1cced47, entries=586128, sequenceid=262185162, filesize=8.9 M 2020-11-17 09:19:23,020 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.29 MB/89437932, heapSize ~128.06 MB/134278432, currentSize=69.70 KB/71373 for 73aae5715542e1a5370f7540d2117abc in 1365ms, sequenceid=262185162, compaction requested=true 2020-11-17 09:19:36,027 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=170.01 MB, max=1.83 GB, blockCount=44246, accesses=28259109, hits=24217594, hitRatio=85.70%, , cachingAccesses=24269540, cachingHits=24212486, cachingHitsRatio=99.76%, evictions=3539, evicted=0, evictedPerRun=0.0 2020-11-17 09:19:38,659 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 58426 ns. 2020-11-17 09:19:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 51743 ns. 2020-11-17 09:20:14,226 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604719189 with entries=28780, filesize=128.53 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604814167 2020-11-17 09:20:22,482 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=102.78 MB heapSize=128.21 MB 2020-11-17 09:20:22,510 INFO [Thread-2955] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:20:23,312 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.78 MB at sequenceid=252212627 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/c30342eb78b741259cce0dcad8b53b44 2020-11-17 09:20:23,319 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:20:23,353 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/c30342eb78b741259cce0dcad8b53b44, entries=360443, sequenceid=252212627, filesize=7.3 M 2020-11-17 09:20:23,355 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.78 MB/107769750, heapSize ~128.21 MB/134434792, currentSize=229.99 KB/235510 for 82421ce8c5c8b48c573baee83caa8305 in 873ms, sequenceid=252212627, compaction requested=true 2020-11-17 09:20:23,356 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 09:20:23,356 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/694237a18629441782fb6457d339784d, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/cd3636bb036142118043b23354c2e7ff, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/c30342eb78b741259cce0dcad8b53b44] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=19.9 M 2020-11-17 09:20:23,385 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:20:23,389 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:20:23,416 INFO [Thread-2957] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:20:28,819 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#652 average throughput is 58.22 MB/second, slept 6 time(s) and total slept time is 2289 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 09:20:28,891 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into dafdae6ec6b743f89e6526a0b567c291(size=19.9 M), total size for store is 11.3 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 09:20:28,891 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605604823355; duration=5sec 2020-11-17 09:20:28,902 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 17952 ns. 2020-11-17 09:20:28,902 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 31769 ns. 2020-11-17 09:21:05,697 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.65 MB heapSize=128.03 MB 2020-11-17 09:21:05,742 INFO [Thread-2961] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:21:06,569 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.65 MB at sequenceid=263655077 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/f31a598b14bb451283b3a0f0751c1285 2020-11-17 09:21:06,577 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:21:06,593 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f31a598b14bb451283b3a0f0751c1285, entries=394856, sequenceid=263655077, filesize=8.0 M 2020-11-17 09:21:06,596 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.65 MB/104489199, heapSize ~128.03 MB/134244256, currentSize=102.31 KB/104770 for e5dc5c620be19b02666a19c280ff582e in 899ms, sequenceid=263655077, compaction requested=true 2020-11-17 09:21:12,084 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:21:32,228 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:21:42,852 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604814167 with entries=27431, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604902821 2020-11-17 09:21:42,853 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603257728 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605603257728 2020-11-17 09:21:42,863 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603324139 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605603324139 2020-11-17 09:21:42,869 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603441991 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605603441991 2020-11-17 09:21:42,875 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603563978 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605603563978 2020-11-17 09:22:12,494 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.20 MB heapSize=128.55 MB 2020-11-17 09:22:12,551 INFO [Thread-2967] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:22:13,372 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.20 MB at sequenceid=272446196 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/930eac8823074af09e56b6a7dd6f7026 2020-11-17 09:22:13,378 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:22:13,401 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/930eac8823074af09e56b6a7dd6f7026, entries=320974, sequenceid=272446196, filesize=8.6 M 2020-11-17 09:22:13,424 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.20 MB/110311925, heapSize ~128.55 MB/134791088, currentSize=864.54 KB/885289 for 0469ce862509756add4a8cdb00bc465c in 930ms, sequenceid=272446196, compaction requested=true 2020-11-17 09:22:13,426 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 09:22:13,426 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/29ceb6c76f4c4cce94933b9c504fe026, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/5b7556b2e9f44794baa94e2443a64d92, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/930eac8823074af09e56b6a7dd6f7026] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=62.0 M 2020-11-17 09:22:13,486 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:22:13,490 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:22:13,513 INFO [Thread-2969] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:22:24,478 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:22:25,280 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#655 average throughput is 62.04 MB/second, slept 13 time(s) and total slept time is 4419 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:22:25,390 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 5a8b7195855a458aa69fd96c8f798480(size=62.8 M), total size for store is 42.9 G. This selection was in queue for 0sec, and took 11sec to execute. 2020-11-17 09:22:25,390 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605604933425; duration=11sec 2020-11-17 09:22:29,206 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:22:38,558 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:23:03,395 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604902821 with entries=23965, filesize=128.28 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604983351 2020-11-17 09:23:14,783 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:23:39,493 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 424710 ns. 2020-11-17 09:23:39,493 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 215057 ns. 2020-11-17 09:23:39,493 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 66115 ns. 2020-11-17 09:24:07,777 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:24:25,911 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:24:32,543 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.00 MB, max=1.83 GB, blockCount=44222, accesses=28445085, hits=24386637, hitRatio=85.73%, , cachingAccesses=24438567, cachingHits=24381501, cachingHitsRatio=99.77%, evictions=3569, evicted=0, evictedPerRun=0.0 2020-11-17 09:24:39,434 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 82483 ns. 2020-11-17 09:24:39,434 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 36891 ns. 2020-11-17 09:24:50,942 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604983351 with entries=32956, filesize=128.32 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605090902 2020-11-17 09:25:04,298 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.36 MB heapSize=128.05 MB 2020-11-17 09:25:04,348 INFO [Thread-2978] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:25:05,125 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.36 MB at sequenceid=263665200 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/3875072ce4334277937a33a1855241f4 2020-11-17 09:25:05,132 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:25:05,148 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3875072ce4334277937a33a1855241f4, entries=398838, sequenceid=263665200, filesize=8.1 M 2020-11-17 09:25:05,150 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.36 MB/104182456, heapSize ~128.05 MB/134271504, currentSize=514.81 KB/527165 for e5dc5c620be19b02666a19c280ff582e in 851ms, sequenceid=263665200, compaction requested=true 2020-11-17 09:25:05,151 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 09:25:05,151 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c9b6047e152a4eaa80a25470c0ba0b22, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f31a598b14bb451283b3a0f0751c1285, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3875072ce4334277937a33a1855241f4] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=76.1 M 2020-11-17 09:25:05,181 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:25:05,183 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:25:05,204 INFO [Thread-2980] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:25:06,276 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.93 MB heapSize=128.26 MB 2020-11-17 09:25:06,305 INFO [Thread-2982] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:25:06,891 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.93 MB at sequenceid=252218828 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/b9c1d388328f4f8aae506d3c2cee74db 2020-11-17 09:25:06,897 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:25:06,915 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b9c1d388328f4f8aae506d3c2cee74db, entries=305244, sequenceid=252218828, filesize=6.3 M 2020-11-17 09:25:06,917 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.93 MB/112129228, heapSize ~128.26 MB/134490680, currentSize=582.92 KB/596915 for 82421ce8c5c8b48c573baee83caa8305 in 641ms, sequenceid=252218828, compaction requested=true 2020-11-17 09:25:11,389 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:25:20,140 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#657 average throughput is 60.68 MB/second, slept 17 time(s) and total slept time is 5428 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:25:20,211 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 6bc2cda77beb489f96f5daa3d5e5331e(size=77.8 M), total size for store is 49.6 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 09:25:20,211 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605605105150; duration=15sec 2020-11-17 09:25:38,737 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 27867 ns. 2020-11-17 09:25:38,737 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 9605 ns. 2020-11-17 09:26:14,165 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605090902 with entries=25944, filesize=128.31 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605174136 2020-11-17 09:26:20,337 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:27:58,436 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605174136 with entries=31047, filesize=128.25 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605278403 2020-11-17 09:28:06,098 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:28:36,856 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.22 MB heapSize=128.96 MB 2020-11-17 09:28:36,897 INFO [Thread-2993] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:28:37,676 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.22 MB at sequenceid=263674136 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/c3c38fa71fa047d8ba8da3adbd9c0d4f 2020-11-17 09:28:37,683 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:28:37,708 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c3c38fa71fa047d8ba8da3adbd9c0d4f, entries=356639, sequenceid=263674136, filesize=8.1 M 2020-11-17 09:28:37,711 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.22 MB/108238069, heapSize ~128.96 MB/135224872, currentSize=374.51 KB/383495 for e5dc5c620be19b02666a19c280ff582e in 855ms, sequenceid=263674136, compaction requested=true 2020-11-17 09:28:55,115 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.69 MB heapSize=128.00 MB 2020-11-17 09:28:55,149 INFO [Thread-2996] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:28:56,227 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.69 MB at sequenceid=288059955 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/771f75cea18c4309813da84e6ca49ff8 2020-11-17 09:28:56,232 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:28:56,246 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/771f75cea18c4309813da84e6ca49ff8, entries=520879, sequenceid=288059955, filesize=9.1 M 2020-11-17 09:28:56,248 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.69 MB/95094654, heapSize ~128.00 MB/134220648, currentSize=65.76 KB/67339 for 5e5f05205d7e40b0141cd977dd009ba1 in 1133ms, sequenceid=288059955, compaction requested=true 2020-11-17 09:29:16,293 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605278403 with entries=24043, filesize=128.64 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605356266 2020-11-17 09:29:24,159 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.05 MB, max=1.83 GB, blockCount=44220, accesses=28464144, hits=24390951, hitRatio=85.69%, , cachingAccesses=24442883, cachingHits=24385808, cachingHitsRatio=99.77%, evictions=3599, evicted=0, evictedPerRun=0.0 2020-11-17 09:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 09:29:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 208333 ns. 2020-11-17 09:29:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 436400 ns. 2020-11-17 09:29:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 53579 ns. 2020-11-17 09:29:54,090 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.93 MB heapSize=128.06 MB 2020-11-17 09:29:54,133 INFO [Thread-3000] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:29:54,873 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.93 MB at sequenceid=260371005 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/b74704a541394593beeb4d45fd2e6ee5 2020-11-17 09:29:54,878 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:29:54,895 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/b74704a541394593beeb4d45fd2e6ee5, entries=377796, sequenceid=260371005, filesize=7.1 M 2020-11-17 09:29:54,897 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.93 MB/105829836, heapSize ~128.06 MB/134279672, currentSize=150.01 KB/153612 for 06f7ca9534710e37d90d9f8345864259 in 806ms, sequenceid=260371005, compaction requested=true 2020-11-17 09:29:54,898 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 09:29:54,898 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/2f1ee6a8a2b546928d4c2e9bcb9a2d08, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/524669ec5593465f85903a5ab063e843, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/b74704a541394593beeb4d45fd2e6ee5] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=125.7 M 2020-11-17 09:29:54,927 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:29:54,945 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:29:54,970 INFO [Thread-3002] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:30:04,197 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:30:04,199 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:30:06,020 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:30:06,022 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:30:14,292 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:30:14,697 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.41 MB heapSize=128.14 MB 2020-11-17 09:30:14,724 INFO [Thread-3005] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:30:14,976 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.11 MB heapSize=128.25 MB 2020-11-17 09:30:15,012 INFO [Thread-3007] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:30:15,362 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.41 MB at sequenceid=252225536 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/a3767a642f334daaa7c6fa7d85b8d727 2020-11-17 09:30:15,382 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a3767a642f334daaa7c6fa7d85b8d727, entries=310674, sequenceid=252225536, filesize=6.3 M 2020-11-17 09:30:15,383 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.41 MB/111577547, heapSize ~128.14 MB/134362312, currentSize=19.48 KB/19947 for 82421ce8c5c8b48c573baee83caa8305 in 686ms, sequenceid=252225536, compaction requested=true 2020-11-17 09:30:15,384 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 09:30:15,384 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/dafdae6ec6b743f89e6526a0b567c291, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b9c1d388328f4f8aae506d3c2cee74db, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a3767a642f334daaa7c6fa7d85b8d727] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=32.5 M 2020-11-17 09:30:15,413 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:30:15,415 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:30:15,439 INFO [Thread-3009] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:30:15,698 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.11 MB at sequenceid=273218715 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/26ff36ffef374fbe86ab465a24fe0abf 2020-11-17 09:30:15,704 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:30:15,722 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/26ff36ffef374fbe86ab465a24fe0abf, entries=276272, sequenceid=273218715, filesize=8.5 M 2020-11-17 09:30:15,723 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.11 MB/113358202, heapSize ~128.25 MB/134477320, currentSize=14.90 KB/15254 for fc93f6dcc564341dcc983bf30971f73d in 747ms, sequenceid=273218715, compaction requested=true 2020-11-17 09:30:28,978 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#662 average throughput is 49.42 MB/second, slept 30 time(s) and total slept time is 16197 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:30:29,072 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into c1b76172f58f437db90b4219490dbfeb(size=125.7 M), total size for store is 55.5 G. This selection was in queue for 0sec, and took 34sec to execute. 2020-11-17 09:30:29,072 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=11, startTime=1605605394897; duration=34sec 2020-11-17 09:30:31,264 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#665 average throughput is 33.38 MB/second, slept 10 time(s) and total slept time is 11057 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:30:31,329 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into f10b3b5fab514e9992baeac2f5dee9e9(size=32.5 M), total size for store is 11.3 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 09:30:31,329 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605605415384; duration=15sec 2020-11-17 09:30:38,996 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 89583 ns. 2020-11-17 09:30:38,996 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 9914 ns. 2020-11-17 09:30:38,996 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 48037 ns. 2020-11-17 09:30:38,996 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 38787 ns. 2020-11-17 09:30:58,258 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605356266 with entries=31796, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605458224 2020-11-17 09:31:10,291 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:31:44,680 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:32:15,784 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605458224 with entries=24203, filesize=128.53 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605535751 2020-11-17 09:32:48,813 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.00 MB heapSize=128.11 MB 2020-11-17 09:32:48,853 INFO [Thread-3019] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:32:49,619 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.00 MB at sequenceid=263684787 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/485289b1fa2f47cb9f1e02b398a5bbac 2020-11-17 09:32:49,627 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:32:49,667 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/485289b1fa2f47cb9f1e02b398a5bbac, entries=362402, sequenceid=263684787, filesize=7.9 M 2020-11-17 09:32:49,668 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.00 MB/106949932, heapSize ~128.11 MB/134335016, currentSize=45.80 KB/46903 for e5dc5c620be19b02666a19c280ff582e in 855ms, sequenceid=263684787, compaction requested=true 2020-11-17 09:32:49,669 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 09:32:49,669 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6bc2cda77beb489f96f5daa3d5e5331e, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c3c38fa71fa047d8ba8da3adbd9c0d4f, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/485289b1fa2f47cb9f1e02b398a5bbac] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=93.8 M 2020-11-17 09:32:49,705 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:32:49,707 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:32:49,731 INFO [Thread-3021] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:33:08,754 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#667 average throughput is 58.43 MB/second, slept 20 time(s) and total slept time is 6400 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:33:08,835 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 6189794f4d104f81a9cd0c82460188fd(size=95.5 M), total size for store is 49.6 G. This selection was in queue for 0sec, and took 19sec to execute. 2020-11-17 09:33:08,835 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605605569669; duration=19sec 2020-11-17 09:33:18,068 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:34:07,736 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605535751 with entries=33893, filesize=128.34 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605647700 2020-11-17 09:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.40 MB, max=1.83 GB, blockCount=44212, accesses=28520255, hits=24393039, hitRatio=85.53%, , cachingAccesses=24444989, cachingHits=24387882, cachingHitsRatio=99.77%, evictions=3629, evicted=0, evictedPerRun=0.0 2020-11-17 09:34:47,201 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:34:56,066 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:35:27,998 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605647700 with entries=24792, filesize=128.43 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605727958 2020-11-17 09:35:37,946 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.33 MB heapSize=128.44 MB 2020-11-17 09:35:37,981 INFO [Thread-3031] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:35:38,741 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.33 MB at sequenceid=272480158 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/ca6669748de84e2b83ebd45c6ac61d5e 2020-11-17 09:35:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 202335 ns. 2020-11-17 09:35:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 414723 ns. 2020-11-17 09:35:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 52222 ns. 2020-11-17 09:35:38,751 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:35:38,771 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/ca6669748de84e2b83ebd45c6ac61d5e, entries=317568, sequenceid=272480158, filesize=8.6 M 2020-11-17 09:35:38,774 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.33 MB/110446470, heapSize ~128.44 MB/134675424, currentSize=589.04 KB/603173 for 0469ce862509756add4a8cdb00bc465c in 828ms, sequenceid=272480158, compaction requested=true 2020-11-17 09:35:44,446 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.78 MB heapSize=128.02 MB 2020-11-17 09:35:44,481 INFO [Thread-3033] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:35:45,625 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.78 MB at sequenceid=291924907 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/67c75eef8e1548468a550efb0b6cf816 2020-11-17 09:35:45,630 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:35:45,647 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/67c75eef8e1548468a550efb0b6cf816, entries=545544, sequenceid=291924907, filesize=8.9 M 2020-11-17 09:35:45,656 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~88.78 MB/93092738, heapSize ~128.02 MB/134235416, currentSize=55.87 KB/57214 for bdaa31a63b7f3d06d9909b7560333e79 in 1210ms, sequenceid=291924907, compaction requested=true 2020-11-17 09:35:45,657 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-17 09:35:45,658 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/8961ddb0a05c4154a3f8cb6707b5cf5a, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/aad78c6177274ff2b32c71b8be62b898, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/343d2679fdc54cc2bd1909c1c612e0eb, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/3e5914bf38b84441a8f43f5b5ee67c60, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/67c75eef8e1548468a550efb0b6cf816] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp, totalSize=423.7 M 2020-11-17 09:35:45,803 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:35:45,806 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:35:45,817 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:35:45,831 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:35:45,849 INFO [Thread-3035] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:35:48,723 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 42613 ns. 2020-11-17 09:35:48,724 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 10369 ns. 2020-11-17 09:35:50,563 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:35:56,491 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.28 MB heapSize=128.05 MB 2020-11-17 09:35:56,526 INFO [Thread-3037] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:35:57,419 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.28 MB at sequenceid=252232938 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/a071a4bff47a4c2fbac4c7dbc9f07700 2020-11-17 09:35:57,428 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:35:57,450 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a071a4bff47a4c2fbac4c7dbc9f07700, entries=311168, sequenceid=252232938, filesize=6.4 M 2020-11-17 09:35:57,452 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.28 MB/111443204, heapSize ~128.05 MB/134265704, currentSize=35.83 KB/36694 for 82421ce8c5c8b48c573baee83caa8305 in 961ms, sequenceid=252232938, compaction requested=true 2020-11-17 09:35:59,209 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:35:59,843 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:35:59,937 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:01,145 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:01,150 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:02,938 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:03,020 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:03,404 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:04,232 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:07,274 INFO [DataStreamer for file /hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/5072ee16aff54093abfc221ac6f688f4] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:10,017 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:13,989 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:14,073 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:14,075 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:14,425 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:14,570 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:14,703 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:14,814 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:15,338 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:15,550 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:15,555 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:16,234 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:16,349 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:16,467 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:16,679 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:17,142 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:20,326 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:20,329 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:20,330 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:20,333 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:21,732 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:23,452 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:23,457 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:24,787 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:24,891 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:27,921 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:27,923 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:27,926 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:27,928 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:27,939 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:28,292 INFO [DataStreamer for file /hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/5072ee16aff54093abfc221ac6f688f4] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:28,369 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:28,371 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:28,785 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:28,788 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:30,627 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:30,631 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:31,169 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:31,173 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:33,878 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:33,880 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:35,725 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:35,727 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:35,845 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:36,461 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:38,405 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:38,407 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:38,524 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:38,527 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:39,007 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 62920 ns. 2020-11-17 09:36:39,007 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 37523 ns. 2020-11-17 09:36:39,939 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:39,942 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:39,946 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:40,024 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:40,026 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:43,514 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:43,516 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:43,519 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:43,521 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:43,534 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:44,606 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:44,608 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:45,133 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:45,136 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:45,138 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:45,234 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:45,236 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:48,675 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.98 MB heapSize=128.07 MB 2020-11-17 09:36:48,719 INFO [Thread-3044] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:49,476 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.06 MB heapSize=128.06 MB 2020-11-17 09:36:49,532 INFO [Thread-3046] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:50,050 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.98 MB at sequenceid=262231278 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/bea82ad08a454e2fbf81384aa054005b 2020-11-17 09:36:50,056 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:50,073 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/bea82ad08a454e2fbf81384aa054005b, entries=580655, sequenceid=262231278, filesize=9.0 M 2020-11-17 09:36:50,074 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.98 MB/90157262, heapSize ~128.07 MB/134292736, currentSize=238.17 KB/243890 for 73aae5715542e1a5370f7540d2117abc in 1400ms, sequenceid=262231278, compaction requested=true 2020-11-17 09:36:50,429 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.06 MB at sequenceid=263694830 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/e793cb2a8b1d492c83dc1a64d7fc2830 2020-11-17 09:36:50,447 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e793cb2a8b1d492c83dc1a64d7fc2830, entries=417438, sequenceid=263694830, filesize=7.9 M 2020-11-17 09:36:50,448 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.06 MB/102822438, heapSize ~128.06 MB/134277584, currentSize=268.67 KB/275113 for e5dc5c620be19b02666a19c280ff582e in 972ms, sequenceid=263694830, compaction requested=true 2020-11-17 09:36:50,449 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 09:36:50,449 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7db8e12ac3b24f94b98fbf9254534c7d, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6189794f4d104f81a9cd0c82460188fd, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e793cb2a8b1d492c83dc1a64d7fc2830] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=216.4 M 2020-11-17 09:36:50,480 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:50,486 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:50,510 INFO [Thread-3048] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:57,233 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:57,236 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:58,952 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:58,978 INFO [DataStreamer for file /hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/5072ee16aff54093abfc221ac6f688f4] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:36:59,150 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:37:00,509 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: bdaa31a63b7f3d06d9909b7560333e79#e#compaction#670 average throughput is 49.65 MB/second, slept 49 time(s) and total slept time is 19435 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:37:00,600 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:37:00,617 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 5 file(s) in e of bdaa31a63b7f3d06d9909b7560333e79 into 5072ee16aff54093abfc221ac6f688f4(size=394.0 M), total size for store is 31.8 G. This selection was in queue for 0sec, and took 1mins, 14sec to execute. 2020-11-17 09:37:00,618 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., storeName=e, priority=10, startTime=1605605745657; duration=1mins, 14sec 2020-11-17 09:37:08,448 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605727958 with entries=30159, filesize=128.26 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605828411 2020-11-17 09:37:08,448 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603679943 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605603679943 2020-11-17 09:37:08,455 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603820604 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605603820604 2020-11-17 09:37:08,476 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605603957370 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605603957370 2020-11-17 09:37:08,489 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604061703 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605604061703 2020-11-17 09:37:08,494 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604209438 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605604209438 2020-11-17 09:37:08,500 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604347614 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605604347614 2020-11-17 09:37:08,508 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604469887 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605604469887 2020-11-17 09:37:08,516 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604583861 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605604583861 2020-11-17 09:37:08,525 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604719189 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605604719189 2020-11-17 09:37:08,534 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604814167 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605604814167 2020-11-17 09:37:08,637 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604902821 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605604902821 2020-11-17 09:37:08,648 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605604983351 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605604983351 2020-11-17 09:37:08,659 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605090902 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605605090902 2020-11-17 09:37:08,665 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605174136 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605605174136 2020-11-17 09:37:11,846 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:37:19,362 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/1b93fb7948d5442eaebaed7a20d8f735] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:37:37,067 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#674 average throughput is 54.08 MB/second, slept 49 time(s) and total slept time is 19005 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:37:37,143 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 1b93fb7948d5442eaebaed7a20d8f735(size=219.2 M), total size for store is 49.6 G. This selection was in queue for 0sec, and took 46sec to execute. 2020-11-17 09:37:37,144 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605605810448; duration=46sec 2020-11-17 09:37:58,686 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:38:27,749 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:38:33,627 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:38:35,586 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605828411 with entries=26716, filesize=128.13 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605915516 2020-11-17 09:38:48,517 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:38:52,643 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:38:57,854 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:39:07,826 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:39:12,113 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:39:16,923 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:39:25,223 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:39:29,336 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:39:33,963 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=177.15 MB, max=1.83 GB, blockCount=43988, accesses=29054632, hits=24825618, hitRatio=85.44%, , cachingAccesses=24877366, cachingHits=24820158, cachingHitsRatio=99.77%, evictions=3659, evicted=0, evictedPerRun=0.0 2020-11-17 09:40:14,818 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605915516 with entries=30637, filesize=128.18 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606014776 2020-11-17 09:40:22,984 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:40:27,286 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:40:31,122 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.53 MB heapSize=128.43 MB 2020-11-17 09:40:31,161 INFO [Thread-3062] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:40:32,063 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.53 MB at sequenceid=263704089 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a2e5d6cb6f8b47cdac4359ab89db296c 2020-11-17 09:40:32,087 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a2e5d6cb6f8b47cdac4359ab89db296c, entries=358916, sequenceid=263704089, filesize=8.2 M 2020-11-17 09:40:32,090 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.53 MB/107513472, heapSize ~128.43 MB/134667504, currentSize=1.26 MB/1326207 for e5dc5c620be19b02666a19c280ff582e in 968ms, sequenceid=263704089, compaction requested=true 2020-11-17 09:40:41,639 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:40:45,145 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:40:52,151 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.07 MB heapSize=128.02 MB 2020-11-17 09:40:52,190 INFO [Thread-3065] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:40:53,009 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.07 MB at sequenceid=260400102 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/ceb02f0aa5c347ffa2e23fcac52ee8d1 2020-11-17 09:40:53,037 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/ceb02f0aa5c347ffa2e23fcac52ee8d1, entries=374244, sequenceid=260400102, filesize=7.1 M 2020-11-17 09:40:53,040 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.07 MB/105978699, heapSize ~128.02 MB/134239272, currentSize=68.13 KB/69765 for 06f7ca9534710e37d90d9f8345864259 in 889ms, sequenceid=260400102, compaction requested=true 2020-11-17 09:40:59,209 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 13154 ns. 2020-11-17 09:40:59,209 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 27181 ns. 2020-11-17 09:41:20,476 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.25 MB heapSize=128.00 MB 2020-11-17 09:41:20,514 INFO [Thread-3068] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:41:21,100 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.25 MB at sequenceid=252240049 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/db675820914c4af9a62cecf9304538ea 2020-11-17 09:41:21,107 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:41:21,125 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/db675820914c4af9a62cecf9304538ea, entries=311083, sequenceid=252240049, filesize=6.3 M 2020-11-17 09:41:21,126 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.25 MB/111407438, heapSize ~128.00 MB/134222920, currentSize=166.38 KB/170378 for 82421ce8c5c8b48c573baee83caa8305 in 650ms, sequenceid=252240049, compaction requested=true 2020-11-17 09:41:21,128 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 09:41:21,128 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/f10b3b5fab514e9992baeac2f5dee9e9, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a071a4bff47a4c2fbac4c7dbc9f07700, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/db675820914c4af9a62cecf9304538ea] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=45.2 M 2020-11-17 09:41:21,154 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:41:21,156 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:41:21,178 INFO [Thread-3070] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:41:21,420 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:41:33,373 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:41:33,857 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#678 average throughput is 58.39 MB/second, slept 14 time(s) and total slept time is 5644 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 09:41:33,923 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into dff9c5f5d4004f9ab25fcfdea33d9130(size=45.1 M), total size for store is 11.3 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 09:41:33,924 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605606081127; duration=12sec 2020-11-17 09:41:39,003 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 219891 ns. 2020-11-17 09:41:39,003 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 447567 ns. 2020-11-17 09:41:39,003 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 66602 ns. 2020-11-17 09:41:41,346 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:41:48,522 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606014776 with entries=27750, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606108497 2020-11-17 09:41:49,416 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:42:39,473 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 59768 ns. 2020-11-17 09:42:39,473 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 35598 ns. 2020-11-17 09:43:02,488 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:43:18,079 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:43:20,971 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606108497 with entries=28141, filesize=128.80 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606200939 2020-11-17 09:44:31,702 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.62 MB heapSize=128.00 MB 2020-11-17 09:44:31,740 INFO [Thread-3080] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:44:32,495 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.62 MB at sequenceid=263714033 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/c724cb900c154aa29cc9691168795bea 2020-11-17 09:44:32,501 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:44:32,519 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c724cb900c154aa29cc9691168795bea, entries=351916, sequenceid=263714033, filesize=8.0 M 2020-11-17 09:44:32,521 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.62 MB/107609124, heapSize ~128.00 MB/134221296, currentSize=58.60 KB/60011 for e5dc5c620be19b02666a19c280ff582e in 819ms, sequenceid=263714033, compaction requested=true 2020-11-17 09:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.48 MB, max=1.83 GB, blockCount=44138, accesses=29650174, hits=25408965, hitRatio=85.70%, , cachingAccesses=25460894, cachingHits=25403499, cachingHitsRatio=99.77%, evictions=3689, evicted=0, evictedPerRun=0.0 2020-11-17 09:44:57,237 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:45:03,709 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606200939 with entries=30988, filesize=128.19 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606303671 2020-11-17 09:45:13,035 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:45:26,524 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:45:46,423 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.21 MB heapSize=128.58 MB 2020-11-17 09:45:46,463 INFO [Thread-3086] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:45:47,139 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.21 MB at sequenceid=273238944 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/4a4f4e29f04f46f8b82e4ba79dec4e7a 2020-11-17 09:45:47,144 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:45:47,175 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/4a4f4e29f04f46f8b82e4ba79dec4e7a, entries=279415, sequenceid=273238944, filesize=8.5 M 2020-11-17 09:45:47,178 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.21 MB/113467645, heapSize ~128.58 MB/134824576, currentSize=20.68 KB/21180 for fc93f6dcc564341dcc983bf30971f73d in 755ms, sequenceid=273238944, compaction requested=true 2020-11-17 09:45:47,179 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 09:45:47,179 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/ef3eb15755c94e69a80dcfa9f016e93b, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/26ff36ffef374fbe86ab465a24fe0abf, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/4a4f4e29f04f46f8b82e4ba79dec4e7a] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=43.0 M 2020-11-17 09:45:47,212 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:45:47,218 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:45:47,240 INFO [Thread-3088] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:45:55,869 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#681 average throughput is 62.47 MB/second, slept 9 time(s) and total slept time is 3420 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:45:55,936 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into 24b880f4e5df4338a6a4c55e3b2999fd(size=43.5 M), total size for store is 33.5 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 09:45:55,936 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605606347178; duration=8sec 2020-11-17 09:46:01,518 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:46:09,312 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 12943 ns. 2020-11-17 09:46:09,312 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 37690 ns. 2020-11-17 09:46:21,474 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:46:31,628 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606303671 with entries=27573, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606391587 2020-11-17 09:46:45,091 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:47:06,455 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:47:27,643 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:47:32,282 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.06 MB heapSize=128.01 MB 2020-11-17 09:47:32,311 INFO [Thread-3096] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:47:32,941 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.06 MB at sequenceid=252247897 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/30d89e4efa7d466eb6366559d985ff5a 2020-11-17 09:47:32,949 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:47:32,968 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/30d89e4efa7d466eb6366559d985ff5a, entries=313627, sequenceid=252247897, filesize=6.4 M 2020-11-17 09:47:32,970 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.06 MB/111211772, heapSize ~128.01 MB/134226064, currentSize=16.50 KB/16901 for 82421ce8c5c8b48c573baee83caa8305 in 688ms, sequenceid=252247897, compaction requested=true 2020-11-17 09:47:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 432006 ns. 2020-11-17 09:47:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 62586 ns. 2020-11-17 09:47:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 159978 ns. 2020-11-17 09:48:06,790 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606391587 with entries=28128, filesize=128.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606486739 2020-11-17 09:48:39,223 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 50378 ns. 2020-11-17 09:48:39,223 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 46932 ns. 2020-11-17 09:48:42,808 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.41 MB heapSize=128.03 MB 2020-11-17 09:48:42,845 INFO [Thread-3102] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:48:43,696 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.41 MB at sequenceid=263724572 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/308bf502f03d4281810365f9ff114f39 2020-11-17 09:48:43,702 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:48:43,719 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/308bf502f03d4281810365f9ff114f39, entries=412150, sequenceid=263724572, filesize=7.9 M 2020-11-17 09:48:43,720 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.41 MB/103185989, heapSize ~128.03 MB/134244384, currentSize=150.55 KB/154160 for e5dc5c620be19b02666a19c280ff582e in 912ms, sequenceid=263724572, compaction requested=true 2020-11-17 09:48:43,721 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 09:48:43,721 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a2e5d6cb6f8b47cdac4359ab89db296c, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c724cb900c154aa29cc9691168795bea, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/308bf502f03d4281810365f9ff114f39] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.1 M 2020-11-17 09:48:43,748 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:48:43,751 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:48:43,777 INFO [Thread-3104] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:48:48,881 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#684 average throughput is 59.26 MB/second, slept 5 time(s) and total slept time is 1433 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:48:48,956 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 58713986a6d2403dba2d87b323f5de03(size=25.0 M), total size for store is 49.6 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 09:48:48,957 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605606523721; duration=5sec 2020-11-17 09:49:19,285 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.83 MB heapSize=128.01 MB 2020-11-17 09:49:19,328 INFO [Thread-3107] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:49:20,034 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.83 MB at sequenceid=272514127 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/8b0405f676534842940e86c09c101983 2020-11-17 09:49:20,043 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:49:20,060 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/8b0405f676534842940e86c09c101983, entries=318687, sequenceid=272514127, filesize=8.6 M 2020-11-17 09:49:20,061 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.83 MB/109918450, heapSize ~128.01 MB/134225152, currentSize=14.31 KB/14654 for 0469ce862509756add4a8cdb00bc465c in 776ms, sequenceid=272514127, compaction requested=true 2020-11-17 09:49:20,062 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 09:49:20,062 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/5a8b7195855a458aa69fd96c8f798480, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/ca6669748de84e2b83ebd45c6ac61d5e, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/8b0405f676534842940e86c09c101983] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=80.1 M 2020-11-17 09:49:20,091 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:49:20,096 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:49:20,114 INFO [Thread-3109] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:49:20,376 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:49:33,808 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:49:33,815 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:49:35,236 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#686 average throughput is 61.90 MB/second, slept 16 time(s) and total slept time is 6427 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:49:35,321 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 1489b7683500448586346999662944fe(size=80.7 M), total size for store is 42.9 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 09:49:35,321 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605606560061; duration=15sec 2020-11-17 09:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.15 MB, max=1.83 GB, blockCount=44149, accesses=29684749, hits=25414730, hitRatio=85.62%, , cachingAccesses=25466658, cachingHits=25409244, cachingHitsRatio=99.77%, evictions=3719, evicted=0, evictedPerRun=0.0 2020-11-17 09:49:37,224 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606486739 with entries=26829, filesize=128.19 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606577204 2020-11-17 09:49:44,291 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:50:01,705 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:50:10,178 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:50:39,199 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:51:15,945 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:51:16,511 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606577204 with entries=30408, filesize=129.10 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606676452 2020-11-17 09:51:18,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 31321 ns. 2020-11-17 09:51:18,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 16356 ns. 2020-11-17 09:52:13,271 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.93 MB heapSize=128.07 MB 2020-11-17 09:52:13,312 INFO [Thread-3119] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:14,050 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.93 MB at sequenceid=260429843 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/83c8d7e5192f4bd58e9d946c97c48436 2020-11-17 09:52:14,059 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:14,081 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/83c8d7e5192f4bd58e9d946c97c48436, entries=378562, sequenceid=260429843, filesize=7.1 M 2020-11-17 09:52:14,084 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.93 MB/105835223, heapSize ~128.07 MB/134289104, currentSize=132.32 KB/135494 for 06f7ca9534710e37d90d9f8345864259 in 813ms, sequenceid=260429843, compaction requested=true 2020-11-17 09:52:23,269 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:26,548 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.00 MB heapSize=128.38 MB 2020-11-17 09:52:26,573 INFO [Thread-3121] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:27,403 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.28 MB heapSize=128.00 MB 2020-11-17 09:52:27,443 INFO [Thread-3123] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:27,591 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.00 MB at sequenceid=263733988 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/8d0184efa36a45f7a2d4bf55bf4c2604 2020-11-17 09:52:27,601 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:27,623 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8d0184efa36a45f7a2d4bf55bf4c2604, entries=394286, sequenceid=263733988, filesize=7.9 M 2020-11-17 09:52:27,624 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.00 MB/104856655, heapSize ~128.38 MB/134620424, currentSize=1.07 MB/1117055 for e5dc5c620be19b02666a19c280ff582e in 1077ms, sequenceid=263733988, compaction requested=true 2020-11-17 09:52:28,455 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.28 MB at sequenceid=288124462 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/b67975c9a41d42a48ce4771957f0cf22 2020-11-17 09:52:28,491 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/b67975c9a41d42a48ce4771957f0cf22, entries=527028, sequenceid=288124462, filesize=9.0 M 2020-11-17 09:52:28,492 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.28 MB/94664995, heapSize ~128.00 MB/134218528, currentSize=28.67 KB/29361 for 5e5f05205d7e40b0141cd977dd009ba1 in 1089ms, sequenceid=288124462, compaction requested=true 2020-11-17 09:52:28,494 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 09:52:28,494 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/d81b2885d2884606b73e454af1810d96, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/771f75cea18c4309813da84e6ca49ff8, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/b67975c9a41d42a48ce4771957f0cf22] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=63.8 M 2020-11-17 09:52:28,680 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:28,687 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:28,721 INFO [Thread-3125] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:30,040 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:32,611 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.39 MB heapSize=128.12 MB 2020-11-17 09:52:32,635 INFO [Thread-3127] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:33,033 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606676452 with entries=23452, filesize=128.39 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606752984 2020-11-17 09:52:33,034 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605278403 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605605278403 2020-11-17 09:52:33,044 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605356266 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605605356266 2020-11-17 09:52:33,061 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605458224 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605605458224 2020-11-17 09:52:33,069 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605535751 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605605535751 2020-11-17 09:52:33,076 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605647700 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605605647700 2020-11-17 09:52:33,544 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.39 MB at sequenceid=252254271 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/d4b0c164f2294b60a5d2f171f5c625b2 2020-11-17 09:52:33,577 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d4b0c164f2294b60a5d2f171f5c625b2, entries=310613, sequenceid=252254271, filesize=6.3 M 2020-11-17 09:52:33,578 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.39 MB/111560953, heapSize ~128.12 MB/134340312, currentSize=456.70 KB/467664 for 82421ce8c5c8b48c573baee83caa8305 in 968ms, sequenceid=252254271, compaction requested=true 2020-11-17 09:52:33,579 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 09:52:33,579 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/dff9c5f5d4004f9ab25fcfdea33d9130, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/30d89e4efa7d466eb6366559d985ff5a, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d4b0c164f2294b60a5d2f171f5c625b2] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=57.8 M 2020-11-17 09:52:33,612 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:33,615 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:33,664 INFO [Thread-3129] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:40,709 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:40,712 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:40,852 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:44,083 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:45,624 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#690 average throughput is 36.64 MB/second, slept 9 time(s) and total slept time is 6773 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:52:45,762 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into d79cb4c590404e84aca0eabf7031baf4(size=64.2 M), total size for store is 24.4 G. This selection was in queue for 0sec, and took 17sec to execute. 2020-11-17 09:52:45,763 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=10, startTime=1605606748493; duration=17sec 2020-11-17 09:52:55,341 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#692 average throughput is 43.94 MB/second, slept 19 time(s) and total slept time is 10451 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:52:55,392 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:55,408 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into a8d1e4c7d1ee4d09899ae34fca83771d(size=57.8 M), total size for store is 11.4 G. This selection was in queue for 0sec, and took 21sec to execute. 2020-11-17 09:52:55,409 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605606753579; duration=21sec 2020-11-17 09:52:58,543 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.77 MB heapSize=128.05 MB 2020-11-17 09:52:58,574 INFO [Thread-3132] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:59,764 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.77 MB at sequenceid=262273907 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/dd49b12bdb5d41c4927692135d91dab2 2020-11-17 09:52:59,770 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:59,787 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/dd49b12bdb5d41c4927692135d91dab2, entries=579272, sequenceid=262273907, filesize=8.8 M 2020-11-17 09:52:59,788 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.77 MB/89937468, heapSize ~128.05 MB/134274760, currentSize=42.51 KB/43532 for 73aae5715542e1a5370f7540d2117abc in 1246ms, sequenceid=262273907, compaction requested=true 2020-11-17 09:52:59,789 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 09:52:59,789 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/87c95deb14e444839b10c886c1cced47, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/bea82ad08a454e2fbf81384aa054005b, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/dd49b12bdb5d41c4927692135d91dab2] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=26.7 M 2020-11-17 09:52:59,843 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:59,850 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:52:59,878 INFO [Thread-3134] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:53:04,199 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#694 average throughput is 58.74 MB/second, slept 3 time(s) and total slept time is 423 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:53:04,280 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into d2cd3a4d01d1405e9d069f26fdefd0da(size=27.3 M), total size for store is 12.7 G. This selection was in queue for 0sec, and took 4sec to execute. 2020-11-17 09:53:04,280 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=10, startTime=1605606779788; duration=4sec 2020-11-17 09:53:14,439 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:53:39,516 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 61411 ns. 2020-11-17 09:53:39,516 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 162561 ns. 2020-11-17 09:53:39,517 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 433803 ns. 2020-11-17 09:53:39,517 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 54054 ns. 2020-11-17 09:53:39,517 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 55807 ns. 2020-11-17 09:54:02,293 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606752984 with entries=27226, filesize=128.53 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606842259 2020-11-17 09:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=174.62 MB, max=1.83 GB, blockCount=44106, accesses=29980861, hits=25681007, hitRatio=85.66%, , cachingAccesses=25732925, cachingHits=25675490, cachingHitsRatio=99.78%, evictions=3749, evicted=0, evictedPerRun=0.0 2020-11-17 09:55:34,761 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606842259 with entries=28327, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606934725 2020-11-17 09:55:49,834 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.62 MB heapSize=128.52 MB 2020-11-17 09:55:49,884 INFO [Thread-3147] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:55:50,882 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.62 MB at sequenceid=263742480 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/abe6c04c8e364c1c99566ae78335a0e7 2020-11-17 09:55:50,887 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:55:50,904 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/abe6c04c8e364c1c99566ae78335a0e7, entries=359044, sequenceid=263742480, filesize=8.0 M 2020-11-17 09:55:50,906 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.62 MB/107607021, heapSize ~128.52 MB/134766480, currentSize=1.29 MB/1348121 for e5dc5c620be19b02666a19c280ff582e in 1072ms, sequenceid=263742480, compaction requested=true 2020-11-17 09:55:50,907 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 09:55:50,907 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/58713986a6d2403dba2d87b323f5de03, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8d0184efa36a45f7a2d4bf55bf4c2604, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/abe6c04c8e364c1c99566ae78335a0e7] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.0 M 2020-11-17 09:55:50,943 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:55:50,947 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:55:50,973 INFO [Thread-3149] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:55:59,238 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#696 average throughput is 61.11 MB/second, slept 9 time(s) and total slept time is 2874 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 09:55:59,307 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into ec17d289a6514b0cbf97a0a0362e0be4(size=42.4 M), total size for store is 49.6 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 09:55:59,308 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605606950906; duration=8sec 2020-11-17 09:56:19,014 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:56:19,529 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 32453 ns. 2020-11-17 09:56:19,529 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15250 ns. 2020-11-17 09:56:48,009 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:57:01,254 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:57:04,625 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606934725 with entries=26799, filesize=128.26 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607024498 2020-11-17 09:57:04,647 WARN [Close-WAL-Writer-366] asyncfs.FanOutOneBlockAsyncDFSOutputHelper: complete file /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606934725 not finished, retry = 0 2020-11-17 09:57:20,481 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.80 MB heapSize=128.43 MB 2020-11-17 09:57:20,527 INFO [Thread-3156] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:57:21,165 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.80 MB at sequenceid=252260454 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/76cde76462444f7d89c6fa80a9462d8d 2020-11-17 09:57:21,170 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:57:21,185 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/76cde76462444f7d89c6fa80a9462d8d, entries=309431, sequenceid=252260454, filesize=6.3 M 2020-11-17 09:57:21,187 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.80 MB/111985439, heapSize ~128.43 MB/134667792, currentSize=507.21 KB/519379 for 82421ce8c5c8b48c573baee83caa8305 in 706ms, sequenceid=252260454, compaction requested=true 2020-11-17 09:57:58,878 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:58:34,815 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 09:58:36,937 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607024498 with entries=28608, filesize=128.26 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607116878 2020-11-17 09:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=174.30 MB, max=1.83 GB, blockCount=44112, accesses=30002804, hits=25694717, hitRatio=85.64%, , cachingAccesses=25746643, cachingHits=25689198, cachingHitsRatio=99.78%, evictions=3779, evicted=0, evictedPerRun=0.0 2020-11-17 09:59:38,918 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 57192 ns. 2020-11-17 09:59:38,918 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 230174 ns. 2020-11-17 09:59:38,918 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 450106 ns. 2020-11-17 09:59:38,919 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 55602 ns. 2020-11-17 09:59:38,919 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 36623 ns. 2020-11-17 10:00:05,637 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.33 MB heapSize=128.07 MB 2020-11-17 10:00:05,674 INFO [Thread-3166] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:00:06,514 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.33 MB at sequenceid=263753297 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a4297842819e481bbd58e54bb4f3d657 2020-11-17 10:00:06,521 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:00:06,538 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a4297842819e481bbd58e54bb4f3d657, entries=371347, sequenceid=263753297, filesize=7.9 M 2020-11-17 10:00:06,539 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.33 MB/106252908, heapSize ~128.07 MB/134294528, currentSize=201.71 KB/206548 for e5dc5c620be19b02666a19c280ff582e in 902ms, sequenceid=263753297, compaction requested=true 2020-11-17 10:00:22,526 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607116878 with entries=33185, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607222488 2020-11-17 10:00:49,430 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.68 MB heapSize=128.00 MB 2020-11-17 10:00:49,474 INFO [Thread-3170] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:00:50,232 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.68 MB at sequenceid=273258323 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/ab66f04a28024f0b8204c6a3eb294314 2020-11-17 10:00:50,243 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:00:50,280 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/ab66f04a28024f0b8204c6a3eb294314, entries=278965, sequenceid=273258323, filesize=8.4 M 2020-11-17 10:00:50,291 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~107.68 MB/112905549, heapSize ~128.00 MB/134222560, currentSize=14.05 KB/14386 for fc93f6dcc564341dcc983bf30971f73d in 861ms, sequenceid=273258323, compaction requested=true 2020-11-17 10:01:29,177 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 31485 ns. 2020-11-17 10:01:29,177 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15924 ns. 2020-11-17 10:01:51,506 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607222488 with entries=27226, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607311468 2020-11-17 10:02:02,567 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:02:08,017 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:02:35,688 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.83 MB heapSize=128.01 MB 2020-11-17 10:02:35,731 INFO [Thread-3177] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:02:36,374 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.83 MB at sequenceid=272547881 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/940a3dc1e32e4bb79207d808e41c3733 2020-11-17 10:02:36,380 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:02:36,398 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/940a3dc1e32e4bb79207d808e41c3733, entries=318552, sequenceid=272547881, filesize=8.6 M 2020-11-17 10:02:36,399 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.83 MB/109922042, heapSize ~128.01 MB/134223856, currentSize=41.19 KB/42175 for 0469ce862509756add4a8cdb00bc465c in 711ms, sequenceid=272547881, compaction requested=true 2020-11-17 10:03:22,650 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:03:42,134 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:03:51,546 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.30 MB heapSize=128.00 MB 2020-11-17 10:03:51,588 INFO [Thread-3182] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:03:52,129 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607311468 with entries=36569, filesize=128.52 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607432107 2020-11-17 10:03:52,403 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.30 MB at sequenceid=260460597 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/57cfefed122f4243af7f036585fe1571 2020-11-17 10:03:52,411 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:03:52,430 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/57cfefed122f4243af7f036585fe1571, entries=384419, sequenceid=260460597, filesize=7.1 M 2020-11-17 10:03:52,436 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.30 MB/105173865, heapSize ~128.00 MB/134219072, currentSize=109.19 KB/111812 for 06f7ca9534710e37d90d9f8345864259 in 890ms, sequenceid=260460597, compaction requested=true 2020-11-17 10:03:52,437 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 10:03:52,437 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/ceb02f0aa5c347ffa2e23fcac52ee8d1, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/83c8d7e5192f4bd58e9d946c97c48436, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/57cfefed122f4243af7f036585fe1571] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=21.3 M 2020-11-17 10:03:52,466 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:03:52,471 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:03:52,503 INFO [Thread-3184] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:03:57,428 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#702 average throughput is 60.77 MB/second, slept 5 time(s) and total slept time is 1491 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 10:03:57,523 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 76f8ff6fcc1e4581b4d953685055ec0f(size=21.4 M), total size for store is 55.5 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 10:03:57,523 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605607432436; duration=5sec 2020-11-17 10:04:03,597 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:04:22,723 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.26 MB heapSize=128.07 MB 2020-11-17 10:04:22,755 INFO [Thread-3187] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:04:23,552 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.26 MB at sequenceid=263764290 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fb5afaf8247a40a691bd32cb386249c1 2020-11-17 10:04:23,559 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:04:23,575 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/fb5afaf8247a40a691bd32cb386249c1, entries=400581, sequenceid=263764290, filesize=7.9 M 2020-11-17 10:04:23,576 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~99.26 MB/104081056, heapSize ~128.07 MB/134288520, currentSize=47.79 KB/48936 for e5dc5c620be19b02666a19c280ff582e in 853ms, sequenceid=263764290, compaction requested=true 2020-11-17 10:04:23,577 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 10:04:23,577 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ec17d289a6514b0cbf97a0a0362e0be4, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a4297842819e481bbd58e54bb4f3d657, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/fb5afaf8247a40a691bd32cb386249c1] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=58.3 M 2020-11-17 10:04:23,603 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:04:23,605 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:04:23,627 INFO [Thread-3189] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:04:35,279 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#704 average throughput is 60.58 MB/second, slept 13 time(s) and total slept time is 4056 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 10:04:35,399 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into efd08b938b164c8c8d22953545d078cd(size=59.9 M), total size for store is 49.6 G. This selection was in queue for 0sec, and took 11sec to execute. 2020-11-17 10:04:35,399 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605607463576; duration=11sec 2020-11-17 10:04:35,922 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.90 MB heapSize=128.55 MB 2020-11-17 10:04:35,944 INFO [Thread-3192] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.83 MB, max=1.83 GB, blockCount=44124, accesses=30028574, hits=25704102, hitRatio=85.60%, , cachingAccesses=25756035, cachingHits=25698578, cachingHitsRatio=99.78%, evictions=3809, evicted=0, evictedPerRun=0.0 2020-11-17 10:04:36,528 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.90 MB at sequenceid=252269896 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/57f2f28bb1cd43b2994cad84d36ba4a2 2020-11-17 10:04:36,556 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/57f2f28bb1cd43b2994cad84d36ba4a2, entries=323208, sequenceid=252269896, filesize=6.5 M 2020-11-17 10:04:36,557 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.90 MB/111043067, heapSize ~128.55 MB/134793568, currentSize=567.98 KB/581607 for 82421ce8c5c8b48c573baee83caa8305 in 635ms, sequenceid=252269896, compaction requested=true 2020-11-17 10:04:36,558 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 10:04:36,558 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a8d1e4c7d1ee4d09899ae34fca83771d, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/76cde76462444f7d89c6fa80a9462d8d, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/57f2f28bb1cd43b2994cad84d36ba4a2] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=70.6 M 2020-11-17 10:04:36,644 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:04:36,648 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:04:36,669 INFO [Thread-3194] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:04:40,025 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:04:55,602 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607432107 with entries=19538, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607495560 2020-11-17 10:04:55,611 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#706 average throughput is 61.53 MB/second, slept 22 time(s) and total slept time is 8605 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 10:04:55,669 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into fb88b29f238144f8a349583d4b391c2a(size=70.6 M), total size for store is 11.4 G. This selection was in queue for 0sec, and took 19sec to execute. 2020-11-17 10:04:55,669 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605607476558; duration=19sec 2020-11-17 10:05:10,231 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:05:14,467 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:05:27,232 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.20 MB heapSize=128.00 MB 2020-11-17 10:05:27,273 INFO [Thread-3198] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:05:28,453 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.20 MB at sequenceid=292003918 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/28dcb596b714470f874df86cd96a2300 2020-11-17 10:05:28,458 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:05:28,475 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/28dcb596b714470f874df86cd96a2300, entries=552611, sequenceid=292003918, filesize=8.6 M 2020-11-17 10:05:28,477 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~88.20 MB/92481334, heapSize ~128.00 MB/134220488, currentSize=54.76 KB/56072 for bdaa31a63b7f3d06d9909b7560333e79 in 1245ms, sequenceid=292003918, compaction requested=true 2020-11-17 10:05:34,452 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:05:38,703 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 213919 ns. 2020-11-17 10:05:38,703 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 55955 ns. 2020-11-17 10:05:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 43449 ns. 2020-11-17 10:05:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 475470 ns. 2020-11-17 10:05:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 50906 ns. 2020-11-17 10:05:42,767 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:06:39,421 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 48378 ns. 2020-11-17 10:06:39,421 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18489 ns. 2020-11-17 10:06:47,727 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607495560 with entries=34112, filesize=128.13 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607607687 2020-11-17 10:06:47,728 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605727958 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605605727958 2020-11-17 10:06:47,752 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605828411 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605605828411 2020-11-17 10:06:47,758 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605605915516 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605605915516 2020-11-17 10:06:47,762 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606014776 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605606014776 2020-11-17 10:06:47,767 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606108497 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605606108497 2020-11-17 10:06:47,771 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606200939 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605606200939 2020-11-17 10:06:47,776 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606303671 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605606303671 2020-11-17 10:06:47,780 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606391587 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605606391587 2020-11-17 10:06:47,785 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606486739 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605606486739 2020-11-17 10:06:47,790 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606577204 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605606577204 2020-11-17 10:07:53,790 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.82 MB heapSize=128.04 MB 2020-11-17 10:07:53,851 INFO [Thread-3209] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:07:54,745 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.53 MB at sequenceid=263773103 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/56d450cc59254a13b3951232ed5ebe90 2020-11-17 10:07:54,834 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:07:54,853 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/56d450cc59254a13b3951232ed5ebe90, entries=364888, sequenceid=263773103, filesize=8.1 M 2020-11-17 10:07:54,856 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.53 MB/107514239, heapSize ~128.85 MB/135108128, currentSize=923.12 KB/945273 for e5dc5c620be19b02666a19c280ff582e in 1066ms, sequenceid=263773103, compaction requested=true 2020-11-17 10:08:03,898 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607607687 with entries=23164, filesize=128.27 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607683874 2020-11-17 10:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.32 MB, max=1.83 GB, blockCount=44133, accesses=30308530, hits=25965134, hitRatio=85.67%, , cachingAccesses=26017089, cachingHits=25959610, cachingHitsRatio=99.78%, evictions=3839, evicted=0, evictedPerRun=0.0 2020-11-17 10:10:18,209 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.96 MB heapSize=128.00 MB 2020-11-17 10:10:18,260 INFO [Thread-3216] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:10:19,515 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.96 MB at sequenceid=262319946 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/1861de47d5cf4623b0023ad19cecf825 2020-11-17 10:10:19,522 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:10:19,540 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/1861de47d5cf4623b0023ad19cecf825, entries=577375, sequenceid=262319946, filesize=8.8 M 2020-11-17 10:10:19,542 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.96 MB/90132001, heapSize ~128.00 MB/134219168, currentSize=454.02 KB/464912 for 73aae5715542e1a5370f7540d2117abc in 1333ms, sequenceid=262319946, compaction requested=true 2020-11-17 10:10:20,588 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607683874 with entries=41949, filesize=128.37 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607820566 2020-11-17 10:10:30,540 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:10:39,558 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 58100 ns. 2020-11-17 10:10:39,559 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 44162 ns. 2020-11-17 10:11:38,979 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 43922 ns. 2020-11-17 10:11:38,979 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 198324 ns. 2020-11-17 10:11:38,980 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 440724 ns. 2020-11-17 10:11:41,690 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.95 MB heapSize=128.00 MB 2020-11-17 10:11:41,739 INFO [Thread-3222] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:11:42,449 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.95 MB at sequenceid=263782636 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/8aac79a000d34938a82598ca414f55ac 2020-11-17 10:11:42,454 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:11:42,470 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8aac79a000d34938a82598ca414f55ac, entries=361555, sequenceid=263782636, filesize=8.1 M 2020-11-17 10:11:42,472 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.95 MB/106906792, heapSize ~128.00 MB/134218304, currentSize=23.21 KB/23772 for e5dc5c620be19b02666a19c280ff582e in 782ms, sequenceid=263782636, compaction requested=true 2020-11-17 10:11:42,473 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 10:11:42,473 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/efd08b938b164c8c8d22953545d078cd, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/56d450cc59254a13b3951232ed5ebe90, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8aac79a000d34938a82598ca414f55ac] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=76.1 M 2020-11-17 10:11:42,505 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:11:42,507 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:11:42,534 INFO [Thread-3224] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:11:48,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15564 ns. 2020-11-17 10:11:48,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 24752 ns. 2020-11-17 10:11:51,290 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:11:57,481 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:11:57,645 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#711 average throughput is 60.22 MB/second, slept 15 time(s) and total slept time is 5071 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 10:11:57,701 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 9a5df7cf17ae4db9994c20548d0aa818(size=77.5 M), total size for store is 49.7 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 10:11:57,702 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605607902472; duration=15sec 2020-11-17 10:12:03,783 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:12:13,139 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:12:17,395 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607820566 with entries=34743, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607937348 2020-11-17 10:12:19,345 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:12:23,318 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:12:26,674 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:12:48,988 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:12:58,292 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:13:02,635 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:13:17,807 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:13:27,596 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:13:31,846 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:13:42,153 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:13:52,929 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:14:02,722 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:14:12,270 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:14:13,549 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607937348 with entries=35386, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608053498 2020-11-17 10:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.38 MB, max=1.83 GB, blockCount=44198, accesses=30954928, hits=26596659, hitRatio=85.92%, , cachingAccesses=26648684, cachingHits=26591128, cachingHitsRatio=99.78%, evictions=3869, evicted=0, evictedPerRun=0.0 2020-11-17 10:14:37,947 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:14:47,155 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:14:55,417 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:15:07,856 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:10,485 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:14,282 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.88 MB heapSize=128.02 MB 2020-11-17 10:16:14,334 INFO [Thread-3236] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:15,227 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.88 MB at sequenceid=273277882 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/c6468eb75b804aa8b255cd776c2d8a4e 2020-11-17 10:16:15,234 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:15,252 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/c6468eb75b804aa8b255cd776c2d8a4e, entries=276109, sequenceid=273277882, filesize=8.4 M 2020-11-17 10:16:15,254 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~107.88 MB/113123000, heapSize ~128.02 MB/134236360, currentSize=46.13 KB/47240 for fc93f6dcc564341dcc983bf30971f73d in 972ms, sequenceid=273277882, compaction requested=true 2020-11-17 10:16:15,256 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 10:16:15,256 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/24b880f4e5df4338a6a4c55e3b2999fd, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/ab66f04a28024f0b8204c6a3eb294314, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/c6468eb75b804aa8b255cd776c2d8a4e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=60.4 M 2020-11-17 10:16:15,296 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:15,301 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:15,321 INFO [Thread-3238] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:16,545 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.14 MB heapSize=128.60 MB 2020-11-17 10:16:16,576 INFO [Thread-3240] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:17,374 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.14 MB at sequenceid=272582379 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/1be33189e9ac4f79a9537d6716ebdfb1 2020-11-17 10:16:17,380 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:17,507 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/1be33189e9ac4f79a9537d6716ebdfb1, entries=322588, sequenceid=272582379, filesize=8.7 M 2020-11-17 10:16:17,511 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.14 MB/110243176, heapSize ~128.60 MB/134849712, currentSize=36.56 KB/37439 for 0469ce862509756add4a8cdb00bc465c in 966ms, sequenceid=272582379, compaction requested=true 2020-11-17 10:16:17,518 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 10:16:17,518 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/1489b7683500448586346999662944fe, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/940a3dc1e32e4bb79207d808e41c3733, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/1be33189e9ac4f79a9537d6716ebdfb1] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=98.0 M 2020-11-17 10:16:17,550 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:17,555 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:17,573 INFO [Thread-3242] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:18,909 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.64 MB heapSize=128.23 MB 2020-11-17 10:16:18,945 INFO [Thread-3244] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:19,797 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.64 MB at sequenceid=263794362 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/9fb44b08e850465a9facef9262ae0bc2 2020-11-17 10:16:19,804 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:19,822 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/9fb44b08e850465a9facef9262ae0bc2, entries=412140, sequenceid=263794362, filesize=8.1 M 2020-11-17 10:16:19,824 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~98.64 MB/103435410, heapSize ~128.23 MB/134457792, currentSize=664.42 KB/680369 for e5dc5c620be19b02666a19c280ff582e in 915ms, sequenceid=263794362, compaction requested=true 2020-11-17 10:16:19,873 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.32 MB heapSize=128.00 MB 2020-11-17 10:16:19,912 INFO [Thread-3246] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:20,907 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.32 MB at sequenceid=288189907 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/e29d4e8ca87a4c0ea8d11a40fb6a5ef2 2020-11-17 10:16:20,927 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/e29d4e8ca87a4c0ea8d11a40fb6a5ef2, entries=526290, sequenceid=288189907, filesize=9.0 M 2020-11-17 10:16:20,928 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.32 MB/94704649, heapSize ~128.00 MB/134218480, currentSize=33.26 KB/34062 for 5e5f05205d7e40b0141cd977dd009ba1 in 1055ms, sequenceid=288189907, compaction requested=true 2020-11-17 10:16:36,088 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:38,029 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608053498 with entries=43594, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608197990 2020-11-17 10:16:38,030 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606676452 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605606676452 2020-11-17 10:16:38,041 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606752984 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605606752984 2020-11-17 10:16:38,050 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606842259 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605606842259 2020-11-17 10:16:38,056 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605606934725 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605606934725 2020-11-17 10:16:38,060 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607024498 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605607024498 2020-11-17 10:16:38,064 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607116878 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605607116878 2020-11-17 10:16:38,069 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607222488 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605607222488 2020-11-17 10:16:38,804 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#713 average throughput is 32.15 MB/second, slept 15 time(s) and total slept time is 16360 ms. 1 active operations remaining, total limit is 57.69 MB/second 2020-11-17 10:16:38,863 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 47185 ns. 2020-11-17 10:16:38,863 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 43206 ns. 2020-11-17 10:16:38,869 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into e467fb37ae3d4509b8f7719bdd2d81e9(size=61.1 M), total size for store is 33.5 G. This selection was in queue for 0sec, and took 23sec to execute. 2020-11-17 10:16:38,870 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605608175255; duration=23sec 2020-11-17 10:16:46,077 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:46,079 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:16:47,750 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#715 average throughput is 37.84 MB/second, slept 22 time(s) and total slept time is 18736 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 10:16:47,805 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into b45f9cd92b7140b08842a559b66de5e7(size=98.5 M), total size for store is 42.9 G. This selection was in queue for 0sec, and took 30sec to execute. 2020-11-17 10:16:47,805 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605608177511; duration=30sec 2020-11-17 10:16:48,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 8977 ns. 2020-11-17 10:16:48,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 33302 ns. 2020-11-17 10:16:52,262 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:17:39,661 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 472335 ns. 2020-11-17 10:17:39,662 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 49771 ns. 2020-11-17 10:17:39,662 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 185170 ns. 2020-11-17 10:17:45,706 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=104.84 MB heapSize=128.00 MB 2020-11-17 10:17:45,735 INFO [Thread-3253] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:17:46,499 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.84 MB at sequenceid=252286434 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/fb6d4bbda10241c18c12fa0097bf6f37 2020-11-17 10:17:46,504 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:17:46,523 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/fb6d4bbda10241c18c12fa0097bf6f37, entries=329863, sequenceid=252286434, filesize=6.8 M 2020-11-17 10:17:46,525 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.84 MB/109933664, heapSize ~128.00 MB/134218216, currentSize=12.49 KB/12789 for 82421ce8c5c8b48c573baee83caa8305 in 818ms, sequenceid=252286434, compaction requested=true 2020-11-17 10:18:02,461 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:18:14,630 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:18:55,815 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:19:06,053 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608197990 with entries=44932, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608346026 2020-11-17 10:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.70 MB, max=1.83 GB, blockCount=44195, accesses=31078890, hits=26689930, hitRatio=85.88%, , cachingAccesses=26741960, cachingHits=26684383, cachingHitsRatio=99.78%, evictions=3899, evicted=0, evictedPerRun=0.0 2020-11-17 10:20:03,421 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.47 MB heapSize=128.27 MB 2020-11-17 10:20:03,470 INFO [Thread-3261] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:20:04,187 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.47 MB at sequenceid=263803770 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/c24659427cf14e7685c8fa81a0829805 2020-11-17 10:20:04,193 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:20:04,211 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c24659427cf14e7685c8fa81a0829805, entries=344055, sequenceid=263803770, filesize=8.1 M 2020-11-17 10:20:04,212 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.47 MB/108497629, heapSize ~128.27 MB/134501376, currentSize=703.36 KB/720240 for e5dc5c620be19b02666a19c280ff582e in 791ms, sequenceid=263803770, compaction requested=true 2020-11-17 10:20:04,214 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 10:20:04,214 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/9a5df7cf17ae4db9994c20548d0aa818, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/9fb44b08e850465a9facef9262ae0bc2, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c24659427cf14e7685c8fa81a0829805] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=93.7 M 2020-11-17 10:20:04,249 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:20:04,251 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:20:04,278 INFO [Thread-3263] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:20:19,947 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:20:22,862 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#720 average throughput is 59.84 MB/second, slept 19 time(s) and total slept time is 6451 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 10:20:22,979 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 2511a60bb336409c8dccfe50e40f275a(size=95.6 M), total size for store is 49.7 G. This selection was in queue for 0sec, and took 18sec to execute. 2020-11-17 10:20:22,979 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605608404213; duration=18sec 2020-11-17 10:20:41,944 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:20:55,389 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:21:05,270 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:21:11,551 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608346026 with entries=38164, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608471496 2020-11-17 10:21:39,304 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 42245 ns. 2020-11-17 10:21:39,304 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 54973 ns. 2020-11-17 10:21:58,728 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 37323 ns. 2020-11-17 10:21:58,728 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 9608 ns. 2020-11-17 10:22:24,719 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=98.64 MB heapSize=128.59 MB 2020-11-17 10:22:24,758 INFO [Thread-3273] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:22:25,664 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.64 MB at sequenceid=260509743 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/58b8d7b8eb164064b91ef123631d8b4d 2020-11-17 10:22:25,670 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:22:25,689 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/58b8d7b8eb164064b91ef123631d8b4d, entries=413758, sequenceid=260509743, filesize=7.9 M 2020-11-17 10:22:25,690 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.64 MB/103431485, heapSize ~128.59 MB/134841416, currentSize=32.55 KB/33336 for 06f7ca9534710e37d90d9f8345864259 in 971ms, sequenceid=260509743, compaction requested=true 2020-11-17 10:23:12,793 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:23:15,201 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608471496 with entries=38031, filesize=128.14 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608595139 2020-11-17 10:23:15,202 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607311468 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605607311468 2020-11-17 10:23:15,220 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607432107 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605607432107 2020-11-17 10:23:18,077 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:23:38,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 433697 ns. 2020-11-17 10:23:38,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 184628 ns. 2020-11-17 10:23:38,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 55597 ns. 2020-11-17 10:23:59,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:24:25,916 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:24:27,439 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.01 MB heapSize=128.05 MB 2020-11-17 10:24:27,473 INFO [Thread-3279] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:24:28,363 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.01 MB at sequenceid=263814860 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/01f5312edcfc4f328ac32d63b0d774ec 2020-11-17 10:24:28,384 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/01f5312edcfc4f328ac32d63b0d774ec, entries=375978, sequenceid=263814860, filesize=8.0 M 2020-11-17 10:24:28,386 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.01 MB/105921026, heapSize ~128.05 MB/134267248, currentSize=25.39 KB/26002 for e5dc5c620be19b02666a19c280ff582e in 947ms, sequenceid=263814860, compaction requested=true 2020-11-17 10:24:32,609 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.72 MB, max=1.83 GB, blockCount=44195, accesses=31369933, hits=26962870, hitRatio=85.95%, , cachingAccesses=27014913, cachingHits=26957309, cachingHitsRatio=99.79%, evictions=3929, evicted=0, evictedPerRun=0.0 2020-11-17 10:25:40,244 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608595139 with entries=43392, filesize=128.09 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608740193 2020-11-17 10:26:17,452 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.82 MB heapSize=128.16 MB 2020-11-17 10:26:17,494 INFO [Thread-3286] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:26:18,680 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.82 MB at sequenceid=262361889 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/fde89046dfb44c3cba749c751664af5b 2020-11-17 10:26:18,687 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:26:18,705 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/fde89046dfb44c3cba749c751664af5b, entries=578145, sequenceid=262361889, filesize=8.9 M 2020-11-17 10:26:18,707 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.82 MB/89992233, heapSize ~128.16 MB/134390408, currentSize=264.47 KB/270819 for 73aae5715542e1a5370f7540d2117abc in 1256ms, sequenceid=262361889, compaction requested=true 2020-11-17 10:26:18,708 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 10:26:18,708 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/d2cd3a4d01d1405e9d069f26fdefd0da, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/1861de47d5cf4623b0023ad19cecf825, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/fde89046dfb44c3cba749c751664af5b] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=45.1 M 2020-11-17 10:26:18,740 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:26:18,746 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:26:18,785 INFO [Thread-3288] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:26:21,378 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:26:24,899 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:26:26,445 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#724 average throughput is 54.83 MB/second, slept 4 time(s) and total slept time is 792 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 10:26:26,552 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into abc2592e3cd647aabcb0438c86ffa65b(size=45.8 M), total size for store is 12.7 G. This selection was in queue for 0sec, and took 7sec to execute. 2020-11-17 10:26:26,552 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=10, startTime=1605608778707; duration=7sec 2020-11-17 10:26:29,829 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:26:39,626 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 62777 ns. 2020-11-17 10:26:39,627 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 51557 ns. 2020-11-17 10:27:09,260 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 30736 ns. 2020-11-17 10:27:09,261 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 10909 ns. 2020-11-17 10:27:31,343 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608740193 with entries=33276, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608851303 2020-11-17 10:28:06,157 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:28:39,177 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.13 MB heapSize=128.18 MB 2020-11-17 10:28:39,220 INFO [Thread-3296] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:28:40,028 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.13 MB at sequenceid=263825227 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/94999bce368f40ee9e9a35e404124a2d 2020-11-17 10:28:40,035 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:28:40,058 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/94999bce368f40ee9e9a35e404124a2d, entries=404369, sequenceid=263825227, filesize=8.0 M 2020-11-17 10:28:40,059 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.13 MB/103950422, heapSize ~128.19 MB/134413768, currentSize=113.00 KB/115716 for e5dc5c620be19b02666a19c280ff582e in 882ms, sequenceid=263825227, compaction requested=true 2020-11-17 10:28:40,060 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 10:28:40,060 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2511a60bb336409c8dccfe50e40f275a, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/01f5312edcfc4f328ac32d63b0d774ec, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/94999bce368f40ee9e9a35e404124a2d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=111.6 M 2020-11-17 10:28:40,090 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:28:40,092 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:28:40,112 INFO [Thread-3298] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:29:01,440 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#726 average throughput is 61.54 MB/second, slept 25 time(s) and total slept time is 7910 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 10:29:01,513 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 7b63fc1cc3aa48d886f4abc438c7bd06(size=113.4 M), total size for store is 49.7 G. This selection was in queue for 0sec, and took 21sec to execute. 2020-11-17 10:29:01,513 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605608920059; duration=21sec 2020-11-17 10:29:13,668 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608851303 with entries=30694, filesize=128.21 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608953631 2020-11-17 10:29:25,428 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.18 MB, max=1.83 GB, blockCount=44172, accesses=31402741, hits=26967396, hitRatio=85.88%, , cachingAccesses=27019415, cachingHits=26961799, cachingHitsRatio=99.79%, evictions=3959, evicted=0, evictedPerRun=0.0 2020-11-17 10:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 10:29:38,778 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 49842 ns. 2020-11-17 10:29:38,778 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 415290 ns. 2020-11-17 10:29:38,778 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 162241 ns. 2020-11-17 10:29:43,291 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.28 MB heapSize=128.33 MB 2020-11-17 10:29:43,324 INFO [Thread-3304] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:29:44,026 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.28 MB at sequenceid=272615857 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/dabcb7baea634fb8836f67930d75a610 2020-11-17 10:29:44,031 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:29:44,046 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/dabcb7baea634fb8836f67930d75a610, entries=317611, sequenceid=272615857, filesize=8.6 M 2020-11-17 10:29:44,047 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.28 MB/110392834, heapSize ~128.33 MB/134567616, currentSize=598.07 KB/612426 for 0469ce862509756add4a8cdb00bc465c in 756ms, sequenceid=272615857, compaction requested=true 2020-11-17 10:30:02,143 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:30:58,929 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608953631 with entries=33233, filesize=128.15 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609058894 2020-11-17 10:31:38,818 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.09 MB heapSize=128.08 MB 2020-11-17 10:31:38,850 INFO [Thread-3310] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:31:39,643 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.09 MB at sequenceid=273297725 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/adf14128d3d54e6d8a38f95eeb2f371b 2020-11-17 10:31:39,649 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:31:39,664 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/adf14128d3d54e6d8a38f95eeb2f371b, entries=273995, sequenceid=273297725, filesize=8.5 M 2020-11-17 10:31:39,666 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.09 MB/113341421, heapSize ~128.08 MB/134304576, currentSize=36.16 KB/37029 for fc93f6dcc564341dcc983bf30971f73d in 848ms, sequenceid=273297725, compaction requested=true 2020-11-17 10:31:44,705 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:31:53,384 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.40 MB heapSize=128.12 MB 2020-11-17 10:31:53,414 INFO [Thread-3314] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:31:54,091 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.40 MB at sequenceid=263833671 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/b90303294d6b4a89b8aa5a10470a391c 2020-11-17 10:31:54,097 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:31:54,119 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b90303294d6b4a89b8aa5a10470a391c, entries=342345, sequenceid=263833671, filesize=8.1 M 2020-11-17 10:31:54,120 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.40 MB/108427476, heapSize ~128.12 MB/134342360, currentSize=629.17 KB/644265 for e5dc5c620be19b02666a19c280ff582e in 736ms, sequenceid=263833671, compaction requested=true 2020-11-17 10:32:18,797 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18755 ns. 2020-11-17 10:32:18,797 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 34712 ns. 2020-11-17 10:32:36,125 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609058894 with entries=29835, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609156092 2020-11-17 10:32:39,028 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 45935 ns. 2020-11-17 10:32:39,028 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 37357 ns. 2020-11-17 10:33:09,900 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=104.93 MB heapSize=128.18 MB 2020-11-17 10:33:09,947 INFO [Thread-3320] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:33:10,582 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.93 MB at sequenceid=252305834 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/f1b6e4edf3d244269e886d8b4c14730b 2020-11-17 10:33:10,588 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:33:10,604 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/f1b6e4edf3d244269e886d8b4c14730b, entries=331116, sequenceid=252305834, filesize=6.9 M 2020-11-17 10:33:10,606 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.93 MB/110025401, heapSize ~128.18 MB/134405784, currentSize=178.76 KB/183049 for 82421ce8c5c8b48c573baee83caa8305 in 705ms, sequenceid=252305834, compaction requested=true 2020-11-17 10:33:10,607 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 10:33:10,607 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/fb88b29f238144f8a349583d4b391c2a, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/fb6d4bbda10241c18c12fa0097bf6f37, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/f1b6e4edf3d244269e886d8b4c14730b] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=84.2 M 2020-11-17 10:33:10,633 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:33:10,640 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:33:10,662 INFO [Thread-3322] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:33:18,069 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:33:30,355 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:33:30,357 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:33:33,221 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#731 average throughput is 60.91 MB/second, slept 25 time(s) and total slept time is 10154 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 10:33:33,295 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 2e34f576d131464b9982ddbefa61cbae(size=84.2 M), total size for store is 11.4 G. This selection was in queue for 0sec, and took 22sec to execute. 2020-11-17 10:33:33,295 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605609190606; duration=22sec 2020-11-17 10:34:13,546 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609156092 with entries=29874, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609253508 2020-11-17 10:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.61 MB, max=1.83 GB, blockCount=44161, accesses=31427331, hits=26969696, hitRatio=85.82%, , cachingAccesses=27021714, cachingHits=26964092, cachingHitsRatio=99.79%, evictions=3989, evicted=0, evictedPerRun=0.0 2020-11-17 10:35:34,342 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609253508 with entries=24319, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609334313 2020-11-17 10:35:38,893 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 177314 ns. 2020-11-17 10:35:38,893 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 392675 ns. 2020-11-17 10:35:38,893 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 54416 ns. 2020-11-17 10:36:17,872 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:36:22,599 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:36:26,098 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:36:29,455 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.45 MB heapSize=128.52 MB 2020-11-17 10:36:29,487 INFO [Thread-3333] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:36:30,331 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.45 MB at sequenceid=263845258 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/2b2cb1e94ac24c07b63770346a611470 2020-11-17 10:36:30,337 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:36:30,361 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2b2cb1e94ac24c07b63770346a611470, entries=376171, sequenceid=263845258, filesize=8.1 M 2020-11-17 10:36:30,363 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.45 MB/106381166, heapSize ~128.52 MB/134760104, currentSize=139.51 KB/142863 for e5dc5c620be19b02666a19c280ff582e in 907ms, sequenceid=263845258, compaction requested=true 2020-11-17 10:36:30,363 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 10:36:30,364 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a204e2c26acf42d2af60e900f55b5304, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1b93fb7948d5442eaebaed7a20d8f735, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7b63fc1cc3aa48d886f4abc438c7bd06, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b90303294d6b4a89b8aa5a10470a391c, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2b2cb1e94ac24c07b63770346a611470] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=761.4 M 2020-11-17 10:36:30,424 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:36:30,428 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:36:30,431 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:36:30,433 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:36:30,462 INFO [Thread-3335] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:36:30,508 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:36:52,869 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/57c9b765dec642ad8acdfdc15f3eb4b5] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:37:16,306 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/57c9b765dec642ad8acdfdc15f3eb4b5] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:37:28,771 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 34078 ns. 2020-11-17 10:37:28,771 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 16701 ns. 2020-11-17 10:37:29,261 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609334313 with entries=35411, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609449195 2020-11-17 10:37:39,206 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/57c9b765dec642ad8acdfdc15f3eb4b5] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:02,125 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/57c9b765dec642ad8acdfdc15f3eb4b5] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:14,844 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=99.45 MB heapSize=128.04 MB 2020-11-17 10:38:14,891 INFO [Thread-3346] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:15,719 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.45 MB at sequenceid=260551475 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/41a33a8f74004f9180502cef662712bb 2020-11-17 10:38:15,727 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:15,747 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/41a33a8f74004f9180502cef662712bb, entries=397363, sequenceid=260551475, filesize=7.6 M 2020-11-17 10:38:15,749 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.45 MB/104284677, heapSize ~128.04 MB/134255040, currentSize=200.89 KB/205708 for 06f7ca9534710e37d90d9f8345864259 in 905ms, sequenceid=260551475, compaction requested=true 2020-11-17 10:38:15,750 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 10:38:15,750 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/76f8ff6fcc1e4581b4d953685055ec0f, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/58b8d7b8eb164064b91ef123631d8b4d, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/41a33a8f74004f9180502cef662712bb] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=36.9 M 2020-11-17 10:38:15,793 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:15,798 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:15,822 INFO [Thread-3348] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:22,502 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:30,043 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#735 average throughput is 34.96 MB/second, slept 9 time(s) and total slept time is 9359 ms. 1 active operations remaining, total limit is 65.38 MB/second 2020-11-17 10:38:30,113 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 81b3cd22851b49638483748652cc32e9(size=37.3 M), total size for store is 55.5 G. This selection was in queue for 0sec, and took 14sec to execute. 2020-11-17 10:38:30,114 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605609495749; duration=14sec 2020-11-17 10:38:32,430 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/57c9b765dec642ad8acdfdc15f3eb4b5] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:38,729 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 45453 ns. 2020-11-17 10:38:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 37608 ns. 2020-11-17 10:38:39,704 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:49,378 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:49,381 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:50,611 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:50,613 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:50,718 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:50,726 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:52,198 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:52,200 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:54,887 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:54,889 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:54,982 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:54,984 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:38:56,113 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#733 average throughput is 59.92 MB/second, slept 157 time(s) and total slept time is 49470 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 10:38:56,201 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 5 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 57c9b765dec642ad8acdfdc15f3eb4b5(size=763.4 M), total size for store is 49.7 G. This selection was in queue for 0sec, and took 2mins, 25sec to execute. 2020-11-17 10:38:56,201 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605609390363; duration=2mins, 25sec 2020-11-17 10:38:57,208 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609449195 with entries=26326, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609537188 2020-11-17 10:39:15,132 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.77 MB heapSize=128.55 MB 2020-11-17 10:39:15,178 INFO [Thread-3353] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:39:15,989 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.77 MB at sequenceid=252313778 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/842abffc66e84692a1684a366894c6d9 2020-11-17 10:39:15,996 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:39:16,014 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/842abffc66e84692a1684a366894c6d9, entries=311463, sequenceid=252313778, filesize=6.4 M 2020-11-17 10:39:16,016 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.77 MB/111951281, heapSize ~128.55 MB/134789744, currentSize=247.42 KB/253360 for 82421ce8c5c8b48c573baee83caa8305 in 883ms, sequenceid=252313778, compaction requested=true 2020-11-17 10:39:36,025 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.80 MB, max=1.83 GB, blockCount=44177, accesses=31866992, hits=27259497, hitRatio=85.54%, , cachingAccesses=27311475, cachingHits=27253837, cachingHitsRatio=99.79%, evictions=4019, evicted=0, evictedPerRun=0.0 2020-11-17 10:39:54,117 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=89.07 MB heapSize=128.00 MB 2020-11-17 10:39:54,167 INFO [Thread-3357] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:39:55,249 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=89.07 MB at sequenceid=292094556 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/d20a7c2081504601a41fd8a099d070f1 2020-11-17 10:39:55,255 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:39:55,272 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/d20a7c2081504601a41fd8a099d070f1, entries=541655, sequenceid=292094556, filesize=9.0 M 2020-11-17 10:39:55,274 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~89.07 MB/93394616, heapSize ~128.00 MB/134217760, currentSize=35.40 KB/36251 for bdaa31a63b7f3d06d9909b7560333e79 in 1157ms, sequenceid=292094556, compaction requested=true 2020-11-17 10:40:22,513 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.81 MB heapSize=128.00 MB 2020-11-17 10:40:22,552 INFO [Thread-3360] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:40:23,636 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.81 MB at sequenceid=288256207 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/be79d2dd110144e6b9a37591b4ed2fc3 2020-11-17 10:40:23,642 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:40:23,659 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/be79d2dd110144e6b9a37591b4ed2fc3, entries=519215, sequenceid=288256207, filesize=9.1 M 2020-11-17 10:40:23,660 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.81 MB/95223714, heapSize ~128.00 MB/134218768, currentSize=57.80 KB/59184 for 5e5f05205d7e40b0141cd977dd009ba1 in 1147ms, sequenceid=288256207, compaction requested=true 2020-11-17 10:40:23,661 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 10:40:23,661 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/d79cb4c590404e84aca0eabf7031baf4, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/e29d4e8ca87a4c0ea8d11a40fb6a5ef2, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/be79d2dd110144e6b9a37591b4ed2fc3] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=82.2 M 2020-11-17 10:40:23,746 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:40:23,751 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:40:23,780 INFO [Thread-3362] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:40:28,446 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:40:28,449 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:40:29,260 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.40 MB heapSize=128.07 MB 2020-11-17 10:40:29,296 INFO [Thread-3364] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:40:30,158 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.40 MB at sequenceid=263855224 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/30a194c788664829830f27dafdc4d44d 2020-11-17 10:40:30,164 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:40:30,179 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/30a194c788664829830f27dafdc4d44d, entries=412862, sequenceid=263855224, filesize=7.9 M 2020-11-17 10:40:30,184 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~98.40 MB/103177872, heapSize ~128.07 MB/134295640, currentSize=1.63 MB/1708246 for e5dc5c620be19b02666a19c280ff582e in 924ms, sequenceid=263855224, compaction requested=true 2020-11-17 10:40:34,664 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:40:34,667 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:40:34,804 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:40:36,567 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:40:36,569 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:40:37,715 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#739 average throughput is 56.80 MB/second, slept 13 time(s) and total slept time is 3809 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 10:40:37,774 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into 18c057a8d94840efb68cad7a07e93bba(size=82.8 M), total size for store is 24.4 G. This selection was in queue for 0sec, and took 14sec to execute. 2020-11-17 10:40:37,774 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=10, startTime=1605609623660; duration=14sec 2020-11-17 10:40:37,894 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609537188 with entries=30914, filesize=128.19 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609637864 2020-11-17 10:40:37,894 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607495560 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605607495560 2020-11-17 10:40:37,907 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607607687 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605607607687 2020-11-17 10:40:37,924 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607683874 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605607683874 2020-11-17 10:40:37,930 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607820566 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605607820566 2020-11-17 10:40:37,935 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605607937348 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605607937348 2020-11-17 10:40:37,939 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608053498 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605608053498 2020-11-17 10:40:37,945 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608197990 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605608197990 2020-11-17 10:40:37,950 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608346026 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605608346026 2020-11-17 10:40:37,954 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608471496 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605608471496 2020-11-17 10:40:37,959 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608595139 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605608595139 2020-11-17 10:41:38,732 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 175871 ns. 2020-11-17 10:41:38,733 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 433433 ns. 2020-11-17 10:41:38,733 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 48349 ns. 2020-11-17 10:42:14,732 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609637864 with entries=29644, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609734680 2020-11-17 10:42:19,849 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.89 MB heapSize=128.03 MB 2020-11-17 10:42:19,880 INFO [Thread-3372] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:42:21,043 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.89 MB at sequenceid=262404352 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/d160681f253a463cbc38f16f4b1eb7af 2020-11-17 10:42:21,051 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:42:21,070 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/d160681f253a463cbc38f16f4b1eb7af, entries=575545, sequenceid=262404352, filesize=8.8 M 2020-11-17 10:42:21,071 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.89 MB/90061011, heapSize ~128.03 MB/134247456, currentSize=65.74 KB/67315 for 73aae5715542e1a5370f7540d2117abc in 1222ms, sequenceid=262404352, compaction requested=true 2020-11-17 10:42:28,836 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 13488 ns. 2020-11-17 10:42:28,836 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 25691 ns. 2020-11-17 10:42:36,529 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:42:36,548 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:43:02,561 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:43:05,802 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:43:21,277 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.92 MB heapSize=128.13 MB 2020-11-17 10:43:21,324 INFO [Thread-3376] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:43:22,078 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.92 MB at sequenceid=272650231 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/15b1282bad00431dbaa38bbcf49d1de5 2020-11-17 10:43:22,084 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:43:22,102 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/15b1282bad00431dbaa38bbcf49d1de5, entries=319023, sequenceid=272650231, filesize=8.7 M 2020-11-17 10:43:22,103 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.92 MB/110020639, heapSize ~128.13 MB/134353120, currentSize=40.76 KB/41740 for 0469ce862509756add4a8cdb00bc465c in 826ms, sequenceid=272650231, compaction requested=true 2020-11-17 10:43:22,104 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 10:43:22,104 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/b45f9cd92b7140b08842a559b66de5e7, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/dabcb7baea634fb8836f67930d75a610, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/15b1282bad00431dbaa38bbcf49d1de5] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=115.7 M 2020-11-17 10:43:22,138 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:43:22,143 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:43:22,162 INFO [Thread-3378] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:43:29,821 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:43:38,797 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 76588 ns. 2020-11-17 10:43:38,797 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 40743 ns. 2020-11-17 10:43:43,591 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:43:43,594 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:43:45,609 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#743 average throughput is 57.48 MB/second, slept 23 time(s) and total slept time is 9333 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 10:43:45,700 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 9e5d8bf8d2b64e74be502b52855f0f8f(size=116.3 M), total size for store is 42.9 G. This selection was in queue for 0sec, and took 23sec to execute. 2020-11-17 10:43:45,701 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605609802104; duration=23sec 2020-11-17 10:43:47,057 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609734680 with entries=27935, filesize=128.38 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609827031 2020-11-17 10:43:47,057 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608740193 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605608740193 2020-11-17 10:43:47,066 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608851303 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605608851303 2020-11-17 10:43:47,074 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605608953631 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605608953631 2020-11-17 10:43:58,798 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.16 MB heapSize=128.24 MB 2020-11-17 10:43:58,824 INFO [Thread-3381] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:43:59,551 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.16 MB at sequenceid=263863928 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/10392d991ba440c48e4cc99f7a28d940 2020-11-17 10:43:59,556 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:43:59,572 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/10392d991ba440c48e4cc99f7a28d940, entries=361623, sequenceid=263863928, filesize=8.0 M 2020-11-17 10:43:59,573 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.16 MB/107120305, heapSize ~128.24 MB/134468144, currentSize=2.93 MB/3076442 for e5dc5c620be19b02666a19c280ff582e in 775ms, sequenceid=263863928, compaction requested=true 2020-11-17 10:44:16,463 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=174.59 MB, max=1.83 GB, blockCount=44110, accesses=31957197, hits=27314862, hitRatio=85.47%, , cachingAccesses=27366842, cachingHits=27309183, cachingHitsRatio=99.79%, evictions=4049, evicted=0, evictedPerRun=0.0 2020-11-17 10:44:57,375 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:08,545 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:11,655 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:16,699 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:17,784 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.32 MB heapSize=128.09 MB 2020-11-17 10:45:17,846 INFO [Thread-3386] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:18,431 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.32 MB at sequenceid=252321617 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/14fc67529749414689c59f5f39ce61b6 2020-11-17 10:45:18,452 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/14fc67529749414689c59f5f39ce61b6, entries=311252, sequenceid=252321617, filesize=6.4 M 2020-11-17 10:45:18,453 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.32 MB/111485424, heapSize ~128.09 MB/134313352, currentSize=31.76 KB/32523 for 82421ce8c5c8b48c573baee83caa8305 in 669ms, sequenceid=252321617, compaction requested=true 2020-11-17 10:45:18,454 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 10:45:18,454 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/2e34f576d131464b9982ddbefa61cbae, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/842abffc66e84692a1684a366894c6d9, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/14fc67529749414689c59f5f39ce61b6] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=97.0 M 2020-11-17 10:45:18,497 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:18,500 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:18,523 INFO [Thread-3388] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:18,571 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:20,472 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609827031 with entries=29029, filesize=128.57 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609920442 2020-11-17 10:45:27,994 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:35,107 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:42,813 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:42,872 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:42,885 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:43,843 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:44,649 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:44,777 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:45:45,661 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#746 average throughput is 58.36 MB/second, slept 31 time(s) and total slept time is 13531 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 10:45:45,739 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 3c8b9fd8319f48b7bcb39748831dd159(size=96.7 M), total size for store is 11.4 G. This selection was in queue for 0sec, and took 27sec to execute. 2020-11-17 10:45:45,739 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605609918454; duration=27sec 2020-11-17 10:46:04,995 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:46:15,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:46:21,922 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:46:31,398 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:46:40,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:46:47,985 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:46:50,294 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609920442 with entries=28096, filesize=128.16 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610010227 2020-11-17 10:47:07,121 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:07,600 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.22 MB heapSize=128.52 MB 2020-11-17 10:47:07,657 INFO [Thread-3395] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:08,397 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.22 MB at sequenceid=273317729 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/b1a11180a6364039ad20d7b2fb32edb8 2020-11-17 10:47:08,418 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/b1a11180a6364039ad20d7b2fb32edb8, entries=278514, sequenceid=273317729, filesize=8.5 M 2020-11-17 10:47:08,419 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.22 MB/113474187, heapSize ~128.52 MB/134761200, currentSize=672.85 KB/688999 for fc93f6dcc564341dcc983bf30971f73d in 819ms, sequenceid=273317729, compaction requested=true 2020-11-17 10:47:08,420 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 10:47:08,420 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/e467fb37ae3d4509b8f7719bdd2d81e9, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/adf14128d3d54e6d8a38f95eeb2f371b, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/b1a11180a6364039ad20d7b2fb32edb8] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=78.0 M 2020-11-17 10:47:08,453 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:08,457 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:08,484 INFO [Thread-3397] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:08,560 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:17,773 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:21,944 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:21,946 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:22,735 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.67 MB heapSize=128.21 MB 2020-11-17 10:47:22,773 INFO [Thread-3400] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:23,678 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.67 MB at sequenceid=263872833 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/dfb008cd81194a91b6637d2d6dbab311 2020-11-17 10:47:23,684 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:23,702 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/dfb008cd81194a91b6637d2d6dbab311, entries=354002, sequenceid=263872833, filesize=8.0 M 2020-11-17 10:47:23,704 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.67 MB/107661383, heapSize ~128.21 MB/134440440, currentSize=1.34 MB/1404571 for e5dc5c620be19b02666a19c280ff582e in 969ms, sequenceid=263872833, compaction requested=true 2020-11-17 10:47:23,705 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 10:47:23,705 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/30a194c788664829830f27dafdc4d44d, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/10392d991ba440c48e4cc99f7a28d940, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/dfb008cd81194a91b6637d2d6dbab311] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=23.9 M 2020-11-17 10:47:23,735 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:23,737 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:23,769 INFO [Thread-3402] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:23,807 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:26,917 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#748 average throughput is 52.55 MB/second, slept 19 time(s) and total slept time is 10158 ms. 1 active operations remaining, total limit is 57.69 MB/second 2020-11-17 10:47:26,977 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into 5af2bb826af342c88365b621882697f2(size=78.4 M), total size for store is 33.5 G. This selection was in queue for 0sec, and took 18sec to execute. 2020-11-17 10:47:26,978 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605610028420; duration=18sec 2020-11-17 10:47:29,238 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 21714 ns. 2020-11-17 10:47:29,238 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 22928 ns. 2020-11-17 10:47:30,727 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#750 average throughput is 43.44 MB/second, slept 6 time(s) and total slept time is 3759 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 10:47:30,785 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 949f361a932d4711ae4cfc14c0ea8170(size=24.9 M), total size for store is 49.7 G. This selection was in queue for 0sec, and took 7sec to execute. 2020-11-17 10:47:30,785 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605610043704; duration=7sec 2020-11-17 10:47:36,040 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:47:39,125 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 203915 ns. 2020-11-17 10:47:39,126 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 461421 ns. 2020-11-17 10:47:39,126 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 49341 ns. 2020-11-17 10:47:39,963 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:48:04,421 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:48:17,032 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:48:19,030 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610010227 with entries=27790, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610098997 2020-11-17 10:48:19,031 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609058894 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605609058894 2020-11-17 10:48:19,037 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609156092 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605609156092 2020-11-17 10:48:19,043 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609253508 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605609253508 2020-11-17 10:48:19,060 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609334313 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605609334313 2020-11-17 10:48:23,265 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:48:39,038 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 49948 ns. 2020-11-17 10:48:39,038 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 36896 ns. 2020-11-17 10:48:53,655 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.28 MB, max=1.83 GB, blockCount=44199, accesses=32715250, hits=28026517, hitRatio=85.67%, , cachingAccesses=28078621, cachingHits=28020817, cachingHitsRatio=99.79%, evictions=4079, evicted=0, evictedPerRun=0.0 2020-11-17 10:49:54,434 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.89 MB heapSize=128.01 MB 2020-11-17 10:49:54,469 INFO [Thread-3409] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:49:55,097 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610098997 with entries=28955, filesize=128.53 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610195078 2020-11-17 10:49:55,258 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.89 MB at sequenceid=260582338 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/54a15154cdd144ecbd56479915934e6a 2020-11-17 10:49:55,264 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:49:55,282 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/54a15154cdd144ecbd56479915934e6a, entries=377050, sequenceid=260582338, filesize=7.1 M 2020-11-17 10:49:55,284 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.89 MB/105787555, heapSize ~128.01 MB/134226328, currentSize=31.48 KB/32232 for 06f7ca9534710e37d90d9f8345864259 in 851ms, sequenceid=260582338, compaction requested=true 2020-11-17 10:50:39,124 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.52 MB heapSize=128.28 MB 2020-11-17 10:50:39,161 INFO [Thread-3413] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:50:39,760 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.52 MB at sequenceid=252328647 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/b6f043300b254c2aab2c8e1181fc1799 2020-11-17 10:50:39,766 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:50:39,784 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b6f043300b254c2aab2c8e1181fc1799, entries=311071, sequenceid=252328647, filesize=6.4 M 2020-11-17 10:50:39,785 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.52 MB/111697793, heapSize ~128.28 MB/134507856, currentSize=16.95 KB/17361 for 82421ce8c5c8b48c573baee83caa8305 in 662ms, sequenceid=252328647, compaction requested=true 2020-11-17 10:51:30,479 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610195078 with entries=29224, filesize=128.47 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610290454 2020-11-17 10:51:30,479 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609449195 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605609449195 2020-11-17 10:51:47,090 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:51:53,379 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:51:59,954 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:52:01,114 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.57 MB heapSize=128.11 MB 2020-11-17 10:52:01,192 INFO [Thread-3419] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:52:02,049 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.18 MB at sequenceid=263884573 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/79795d0db88346dfbb7e419b16dfbc69 2020-11-17 10:52:02,072 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/79795d0db88346dfbb7e419b16dfbc69, entries=412252, sequenceid=263884573, filesize=8.0 M 2020-11-17 10:52:02,080 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.18 MB/103994993, heapSize ~128.80 MB/135052168, currentSize=553.62 KB/566902 for e5dc5c620be19b02666a19c280ff582e in 966ms, sequenceid=263884573, compaction requested=true 2020-11-17 10:52:05,132 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:52:39,065 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 26881 ns. 2020-11-17 10:52:39,065 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 25694 ns. 2020-11-17 10:52:51,857 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610290454 with entries=24470, filesize=128.19 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610371802 2020-11-17 10:53:39,625 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 55358 ns. 2020-11-17 10:53:39,626 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 188389 ns. 2020-11-17 10:53:39,626 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 433961 ns. 2020-11-17 10:53:39,626 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 41828 ns. 2020-11-17 10:53:39,626 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 44231 ns. 2020-11-17 10:54:01,718 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:54:32,331 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610371802 with entries=30900, filesize=128.41 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610472280 2020-11-17 10:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=170.32 MB, max=1.83 GB, blockCount=44217, accesses=32984426, hits=28295675, hitRatio=85.78%, , cachingAccesses=28347797, cachingHits=28289975, cachingHitsRatio=99.80%, evictions=4109, evicted=0, evictedPerRun=0.0 2020-11-17 10:56:01,754 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.92 MB heapSize=128.39 MB 2020-11-17 10:56:01,789 INFO [Thread-3429] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:56:02,577 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.92 MB at sequenceid=263894641 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/cce1d5b161ee44e6ac45997c17de387d 2020-11-17 10:56:02,588 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:56:02,615 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/cce1d5b161ee44e6ac45997c17de387d, entries=395627, sequenceid=263894641, filesize=8.0 M 2020-11-17 10:56:02,618 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.92 MB/104774215, heapSize ~128.39 MB/134627720, currentSize=1.81 MB/1895668 for e5dc5c620be19b02666a19c280ff582e in 864ms, sequenceid=263894641, compaction requested=true 2020-11-17 10:56:02,620 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 10:56:02,620 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/949f361a932d4711ae4cfc14c0ea8170, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/79795d0db88346dfbb7e419b16dfbc69, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/cce1d5b161ee44e6ac45997c17de387d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.0 M 2020-11-17 10:56:02,665 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:56:02,667 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:56:02,809 INFO [Thread-3431] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:56:11,148 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610472280 with entries=29645, filesize=128.13 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610571130 2020-11-17 10:56:11,382 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#755 average throughput is 57.71 MB/second, slept 10 time(s) and total slept time is 2929 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 10:56:11,435 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 745255bd17f64b23b191e769a3f8238a(size=42.4 M), total size for store is 49.7 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 10:56:11,435 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605610562619; duration=8sec 2020-11-17 10:56:17,724 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.25 MB heapSize=128.09 MB 2020-11-17 10:56:17,747 INFO [Thread-3433] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:56:18,333 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.25 MB at sequenceid=252335921 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/3082f38cb6d940aba06ff47fb7570c28 2020-11-17 10:56:18,340 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:56:18,357 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/3082f38cb6d940aba06ff47fb7570c28, entries=312295, sequenceid=252335921, filesize=6.3 M 2020-11-17 10:56:18,359 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.25 MB/111408556, heapSize ~128.09 MB/134314960, currentSize=18.97 KB/19422 for 82421ce8c5c8b48c573baee83caa8305 in 636ms, sequenceid=252335921, compaction requested=true 2020-11-17 10:56:18,359 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 10:56:18,360 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/3c8b9fd8319f48b7bcb39748831dd159, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b6f043300b254c2aab2c8e1181fc1799, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/3082f38cb6d940aba06ff47fb7570c28] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=109.4 M 2020-11-17 10:56:18,396 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:56:18,399 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:56:18,431 INFO [Thread-3435] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:56:19,891 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:56:46,666 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.25 MB heapSize=128.37 MB 2020-11-17 10:56:46,720 INFO [Thread-3439] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:56:47,418 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.25 MB at sequenceid=272683942 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/41f7a2e6240d464299c1e372cbd3a164 2020-11-17 10:56:47,424 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:56:47,444 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:56:47,447 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:56:47,450 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/41f7a2e6240d464299c1e372cbd3a164, entries=317676, sequenceid=272683942, filesize=8.6 M 2020-11-17 10:56:47,452 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.25 MB/110365796, heapSize ~128.37 MB/134600832, currentSize=22.88 KB/23432 for 0469ce862509756add4a8cdb00bc465c in 786ms, sequenceid=272683942, compaction requested=true 2020-11-17 10:56:50,429 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#757 average throughput is 56.01 MB/second, slept 29 time(s) and total slept time is 12502 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 10:56:50,510 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into b8a29016bcc3406db017cc9309dcaad4(size=109.1 M), total size for store is 11.4 G. This selection was in queue for 0sec, and took 32sec to execute. 2020-11-17 10:56:50,511 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605610578359; duration=32sec 2020-11-17 10:56:53,931 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:57:01,266 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:57:09,367 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:57:33,414 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610571130 with entries=26163, filesize=128.50 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610653378 2020-11-17 10:57:48,991 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 28052 ns. 2020-11-17 10:57:48,991 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 13169 ns. 2020-11-17 10:57:58,159 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:58:25,292 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.27 MB heapSize=128.00 MB 2020-11-17 10:58:25,327 INFO [Thread-3444] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:58:26,429 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.27 MB at sequenceid=262447137 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/3550cf9ca9ac448484d30b64d260ff1a 2020-11-17 10:58:26,440 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:58:26,456 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/3550cf9ca9ac448484d30b64d260ff1a, entries=583640, sequenceid=262447137, filesize=8.8 M 2020-11-17 10:58:26,457 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.27 MB/89410043, heapSize ~128.00 MB/134220432, currentSize=31.87 KB/32630 for 73aae5715542e1a5370f7540d2117abc in 1165ms, sequenceid=262447137, compaction requested=true 2020-11-17 10:58:26,458 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 10:58:26,458 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/abc2592e3cd647aabcb0438c86ffa65b, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/d160681f253a463cbc38f16f4b1eb7af, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/3550cf9ca9ac448484d30b64d260ff1a] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=63.3 M 2020-11-17 10:58:26,488 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:58:26,507 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:58:26,534 INFO [Thread-3446] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:58:33,835 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:58:36,345 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#760 average throughput is 59.52 MB/second, slept 9 time(s) and total slept time is 1460 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 10:58:36,448 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into d6069e75cd894afb80a17577bcf06d47(size=64.0 M), total size for store is 12.7 G. This selection was in queue for 0sec, and took 9sec to execute. 2020-11-17 10:58:36,448 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=10, startTime=1605610706458; duration=9sec 2020-11-17 10:58:47,339 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:59:03,447 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610653378 with entries=27003, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610743394 2020-11-17 10:59:03,636 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.03 MB heapSize=128.76 MB 2020-11-17 10:59:03,662 INFO [Thread-3449] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:59:04,545 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.03 MB at sequenceid=263902435 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/9679179c4e154d76be09774d98e0102f 2020-11-17 10:59:04,550 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 10:59:04,568 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/9679179c4e154d76be09774d98e0102f, entries=356686, sequenceid=263902435, filesize=7.9 M 2020-11-17 10:59:04,569 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.03 MB/108031729, heapSize ~128.76 MB/135011192, currentSize=1.70 MB/1785653 for e5dc5c620be19b02666a19c280ff582e in 933ms, sequenceid=263902435, compaction requested=true 2020-11-17 10:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=170.45 MB, max=1.83 GB, blockCount=44214, accesses=33045344, hits=28309734, hitRatio=85.67%, , cachingAccesses=28361816, cachingHits=28303978, cachingHitsRatio=99.80%, evictions=4139, evicted=0, evictedPerRun=0.0 2020-11-17 10:59:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 190606 ns. 2020-11-17 10:59:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 70797 ns. 2020-11-17 10:59:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 476331 ns. 2020-11-17 10:59:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 51628 ns. 2020-11-17 10:59:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 36853 ns. 2020-11-17 11:00:48,124 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610743394 with entries=32687, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610848082 2020-11-17 11:00:48,144 WARN [Close-WAL-Writer-403] asyncfs.FanOutOneBlockAsyncDFSOutputHelper: complete file /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610743394 not finished, retry = 0 2020-11-17 11:01:26,967 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.86 MB heapSize=128.02 MB 2020-11-17 11:01:26,999 INFO [Thread-3457] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:01:27,743 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.86 MB at sequenceid=260612963 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/b8dc426fd8df4870a5f80a3d6e684ab5 2020-11-17 11:01:27,751 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:01:27,766 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/b8dc426fd8df4870a5f80a3d6e684ab5, entries=378544, sequenceid=260612963, filesize=7.1 M 2020-11-17 11:01:27,768 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.86 MB/105762554, heapSize ~128.02 MB/134234128, currentSize=22.92 KB/23469 for 06f7ca9534710e37d90d9f8345864259 in 801ms, sequenceid=260612963, compaction requested=true 2020-11-17 11:01:27,769 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 11:01:27,769 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/81b3cd22851b49638483748652cc32e9, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/54a15154cdd144ecbd56479915934e6a, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/b8dc426fd8df4870a5f80a3d6e684ab5] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=51.5 M 2020-11-17 11:01:27,798 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:01:27,805 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:01:27,830 INFO [Thread-3459] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:01:39,705 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#763 average throughput is 58.75 MB/second, slept 12 time(s) and total slept time is 4734 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 11:01:39,773 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:01:39,790 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 8cf8973a5ce44d3c94d06ebbfe08eac6(size=51.9 M), total size for store is 55.6 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 11:01:39,790 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605610887768; duration=12sec 2020-11-17 11:02:02,397 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:02:07,055 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:02:18,962 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:02:20,408 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.20 MB heapSize=128.16 MB 2020-11-17 11:02:20,448 INFO [Thread-3465] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:02:21,037 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.20 MB at sequenceid=252343749 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/85d994ab9a9b4fe09ae765a9f9f79c99 2020-11-17 11:02:21,062 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/85d994ab9a9b4fe09ae765a9f9f79c99, entries=313689, sequenceid=252343749, filesize=6.4 M 2020-11-17 11:02:21,063 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.20 MB/111363391, heapSize ~128.16 MB/134381824, currentSize=23.55 KB/24117 for 82421ce8c5c8b48c573baee83caa8305 in 655ms, sequenceid=252343749, compaction requested=true 2020-11-17 11:02:22,372 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610848082 with entries=29255, filesize=128.16 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610942353 2020-11-17 11:02:24,022 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.94 MB heapSize=128.17 MB 2020-11-17 11:02:24,043 INFO [Thread-3467] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:02:24,773 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.94 MB at sequenceid=273337675 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/158da4fded15440b9247221121998d95 2020-11-17 11:02:24,778 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:02:24,794 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/158da4fded15440b9247221121998d95, entries=277636, sequenceid=273337675, filesize=8.4 M 2020-11-17 11:02:24,795 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~107.94 MB/113179517, heapSize ~128.17 MB/134397248, currentSize=6.58 KB/6742 for fc93f6dcc564341dcc983bf30971f73d in 773ms, sequenceid=273337675, compaction requested=true 2020-11-17 11:02:49,595 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15769 ns. 2020-11-17 11:02:49,596 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 33417 ns. 2020-11-17 11:03:18,081 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:03:18,893 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.88 MB heapSize=128.01 MB 2020-11-17 11:03:18,925 INFO [Thread-3471] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:03:19,696 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.88 MB at sequenceid=263913351 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/3c9d919f72b14c3185896ad83be24f06 2020-11-17 11:03:19,734 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3c9d919f72b14c3185896ad83be24f06, entries=362507, sequenceid=263913351, filesize=8.0 M 2020-11-17 11:03:19,735 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.88 MB/106829596, heapSize ~128.01 MB/134230984, currentSize=138.67 KB/141995 for e5dc5c620be19b02666a19c280ff582e in 842ms, sequenceid=263913351, compaction requested=true 2020-11-17 11:03:19,736 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 11:03:19,736 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/745255bd17f64b23b191e769a3f8238a, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/9679179c4e154d76be09774d98e0102f, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3c9d919f72b14c3185896ad83be24f06] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=58.3 M 2020-11-17 11:03:19,764 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:03:19,767 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:03:19,790 INFO [Thread-3473] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:03:22,924 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:03:32,044 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#767 average throughput is 57.64 MB/second, slept 13 time(s) and total slept time is 5154 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 11:03:32,110 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:03:32,128 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 647c4976de70451c882e0be2dfef5139(size=59.9 M), total size for store is 49.8 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 11:03:32,129 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605610999735; duration=12sec 2020-11-17 11:03:42,130 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:04:03,353 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:04:03,389 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610942353 with entries=30386, filesize=128.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611043366 2020-11-17 11:04:11,323 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.22 MB heapSize=128.00 MB 2020-11-17 11:04:11,350 INFO [Thread-3476] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:04:12,396 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.22 MB at sequenceid=288322178 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/d45fadb7e3354cab815c2fe38435f1de 2020-11-17 11:04:12,405 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:04:12,425 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/d45fadb7e3354cab815c2fe38435f1de, entries=527760, sequenceid=288322178, filesize=9.0 M 2020-11-17 11:04:12,426 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.22 MB/94605444, heapSize ~128.00 MB/134219944, currentSize=114.26 KB/116999 for 5e5f05205d7e40b0141cd977dd009ba1 in 1103ms, sequenceid=288322178, compaction requested=true 2020-11-17 11:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.10 MB, max=1.83 GB, blockCount=44158, accesses=33077642, hits=28319175, hitRatio=85.61%, , cachingAccesses=28371256, cachingHits=28313401, cachingHitsRatio=99.80%, evictions=4169, evicted=0, evictedPerRun=0.0 2020-11-17 11:04:40,236 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:05:10,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:05:26,529 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611043366 with entries=25796, filesize=128.16 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611126488 2020-11-17 11:05:38,751 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 52848 ns. 2020-11-17 11:05:38,751 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 60500 ns. 2020-11-17 11:05:38,751 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 196343 ns. 2020-11-17 11:05:38,752 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 496230 ns. 2020-11-17 11:05:38,752 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 33911 ns. 2020-11-17 11:07:11,153 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:07:11,761 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611126488 with entries=31893, filesize=128.13 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611231711 2020-11-17 11:07:17,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:07:24,982 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=97.98 MB heapSize=128.08 MB 2020-11-17 11:07:25,039 INFO [Thread-3487] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:07:26,050 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.98 MB at sequenceid=263923780 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/3ad44cf0575449b2a3b9ed5c74b2685e 2020-11-17 11:07:26,056 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:07:26,072 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3ad44cf0575449b2a3b9ed5c74b2685e, entries=418932, sequenceid=263923780, filesize=7.9 M 2020-11-17 11:07:26,075 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~97.98 MB/102742999, heapSize ~128.08 MB/134304696, currentSize=251.00 KB/257028 for e5dc5c620be19b02666a19c280ff582e in 1093ms, sequenceid=263923780, compaction requested=true 2020-11-17 11:07:29,952 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:07:58,724 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 12538 ns. 2020-11-17 11:07:58,724 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 38024 ns. 2020-11-17 11:08:30,523 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611231711 with entries=24024, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611310488 2020-11-17 11:08:52,768 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.02 MB heapSize=128.01 MB 2020-11-17 11:08:52,816 INFO [Thread-3492] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:08:53,461 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.02 MB at sequenceid=252352077 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/32506f0a87794770a6360159202bdefb 2020-11-17 11:08:53,467 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:08:53,486 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/32506f0a87794770a6360159202bdefb, entries=314046, sequenceid=252352077, filesize=6.4 M 2020-11-17 11:08:53,487 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.02 MB/111174740, heapSize ~128.01 MB/134227368, currentSize=89.76 KB/91914 for 82421ce8c5c8b48c573baee83caa8305 in 719ms, sequenceid=252352077, compaction requested=true 2020-11-17 11:08:53,489 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 11:08:53,489 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b8a29016bcc3406db017cc9309dcaad4, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/85d994ab9a9b4fe09ae765a9f9f79c99, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/32506f0a87794770a6360159202bdefb] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=121.9 M 2020-11-17 11:08:53,520 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:08:53,524 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:08:53,546 INFO [Thread-3494] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:09:24,638 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:09:25,320 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:09:28,276 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:09:28,278 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:09:29,044 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#771 average throughput is 56.37 MB/second, slept 38 time(s) and total slept time is 16436 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 11:09:29,143 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 53f3df4411644d1f9f121db051afe62f(size=121.5 M), total size for store is 11.4 G. This selection was in queue for 0sec, and took 35sec to execute. 2020-11-17 11:09:29,143 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605611333487; duration=35sec 2020-11-17 11:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.14 MB, max=1.83 GB, blockCount=44177, accesses=33379821, hits=28588891, hitRatio=85.65%, , cachingAccesses=28640989, cachingHits=28583115, cachingHitsRatio=99.80%, evictions=4199, evicted=0, evictedPerRun=0.0 2020-11-17 11:10:24,223 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.67 MB heapSize=128.05 MB 2020-11-17 11:10:24,294 INFO [Thread-3499] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:10:25,124 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.67 MB at sequenceid=272718568 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/dec2f679d7274a658ae29c17ad2369d7 2020-11-17 11:10:25,131 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:10:25,160 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/dec2f679d7274a658ae29c17ad2369d7, entries=321410, sequenceid=272718568, filesize=8.7 M 2020-11-17 11:10:25,164 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.67 MB/109756181, heapSize ~128.05 MB/134274328, currentSize=38.12 KB/39037 for 0469ce862509756add4a8cdb00bc465c in 940ms, sequenceid=272718568, compaction requested=true 2020-11-17 11:10:26,518 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.40 MB heapSize=128.27 MB 2020-11-17 11:10:26,577 INFO [Thread-3501] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:10:27,728 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.40 MB at sequenceid=292176290 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/f6840bb5e87047cbb30f9b4d42899c0f 2020-11-17 11:10:27,767 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/f6840bb5e87047cbb30f9b4d42899c0f, entries=553695, sequenceid=292176290, filesize=8.7 M 2020-11-17 11:10:27,770 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~88.40 MB/92695161, heapSize ~128.27 MB/134503568, currentSize=62.74 KB/64242 for bdaa31a63b7f3d06d9909b7560333e79 in 1251ms, sequenceid=292176290, compaction requested=true 2020-11-17 11:10:27,771 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-17 11:10:27,771 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/28dcb596b714470f874df86cd96a2300, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/d20a7c2081504601a41fd8a099d070f1, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/f6840bb5e87047cbb30f9b4d42899c0f] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp, totalSize=26.4 M 2020-11-17 11:10:27,803 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:10:27,807 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:10:27,836 INFO [Thread-3503] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:10:29,449 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611310488 with entries=36898, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611429413 2020-11-17 11:10:29,449 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609537188 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605609537188 2020-11-17 11:10:29,462 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609637864 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605609637864 2020-11-17 11:10:29,469 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609734680 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605609734680 2020-11-17 11:10:29,476 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609827031 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605609827031 2020-11-17 11:10:29,480 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605609920442 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605609920442 2020-11-17 11:10:29,484 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610010227 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605610010227 2020-11-17 11:10:29,490 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610098997 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605610098997 2020-11-17 11:10:29,505 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610195078 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605610195078 2020-11-17 11:10:29,510 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610290454 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605610290454 2020-11-17 11:10:29,515 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610371802 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605610371802 2020-11-17 11:10:29,520 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610472280 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605610472280 2020-11-17 11:10:29,525 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610571130 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605610571130 2020-11-17 11:10:30,831 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:10:32,418 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: bdaa31a63b7f3d06d9909b7560333e79#e#compaction#774 average throughput is 57.27 MB/second, slept 3 time(s) and total slept time is 883 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 11:10:32,496 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of bdaa31a63b7f3d06d9909b7560333e79 into 535592258538419fb7322a690da9cde3(size=26.7 M), total size for store is 31.9 G. This selection was in queue for 0sec, and took 4sec to execute. 2020-11-17 11:10:32,496 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., storeName=e, priority=11, startTime=1605611427771; duration=4sec 2020-11-17 11:10:39,224 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 49679 ns. 2020-11-17 11:10:39,224 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 63749 ns. 2020-11-17 11:11:12,313 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.59 MB heapSize=128.10 MB 2020-11-17 11:11:12,384 INFO [Thread-3507] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:11:13,172 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.59 MB at sequenceid=263933391 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/e8a0bc7ba8d24b96a37241bd798f981c 2020-11-17 11:11:13,178 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:11:13,195 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e8a0bc7ba8d24b96a37241bd798f981c, entries=367951, sequenceid=263933391, filesize=8.1 M 2020-11-17 11:11:13,196 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.59 MB/106522882, heapSize ~128.10 MB/134320160, currentSize=1.41 MB/1482657 for e5dc5c620be19b02666a19c280ff582e in 883ms, sequenceid=263933391, compaction requested=true 2020-11-17 11:11:13,198 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 11:11:13,198 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/647c4976de70451c882e0be2dfef5139, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3ad44cf0575449b2a3b9ed5c74b2685e, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e8a0bc7ba8d24b96a37241bd798f981c] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=75.9 M 2020-11-17 11:11:13,231 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:11:13,235 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:11:13,261 INFO [Thread-3509] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:11:28,286 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#776 average throughput is 60.29 MB/second, slept 16 time(s) and total slept time is 4121 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 11:11:28,366 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into c223c73cfb1c477ba3353cdd34eb55dc(size=77.8 M), total size for store is 49.8 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 11:11:28,366 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605611473197; duration=15sec 2020-11-17 11:11:38,904 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 221801 ns. 2020-11-17 11:11:38,905 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 479799 ns. 2020-11-17 11:11:38,905 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 55607 ns. 2020-11-17 11:11:39,710 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611429413 with entries=21296, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611499691 2020-11-17 11:12:58,761 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 27745 ns. 2020-11-17 11:12:58,761 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 9474 ns. 2020-11-17 11:13:00,534 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:13:17,806 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:13:18,076 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:13:23,883 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.78 MB heapSize=128.00 MB 2020-11-17 11:13:23,944 INFO [Thread-3519] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:13:24,703 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.78 MB at sequenceid=260644524 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/a0461cdd999d42cfb8228bf0e2aa6f74 2020-11-17 11:13:24,709 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:13:24,732 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/a0461cdd999d42cfb8228bf0e2aa6f74, entries=380212, sequenceid=260644524, filesize=7.2 M 2020-11-17 11:13:24,734 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.78 MB/105671147, heapSize ~128.00 MB/134220944, currentSize=40.25 KB/41214 for 06f7ca9534710e37d90d9f8345864259 in 852ms, sequenceid=260644524, compaction requested=true 2020-11-17 11:13:32,836 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:13:33,720 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611499691 with entries=35216, filesize=128.26 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611613698 2020-11-17 11:13:42,033 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:13:42,522 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.48 MB heapSize=128.11 MB 2020-11-17 11:13:42,558 INFO [Thread-3521] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:13:43,603 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.48 MB at sequenceid=262487702 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/f82dfa223ecf4ee9965af68a568b1fd2 2020-11-17 11:13:43,622 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/f82dfa223ecf4ee9965af68a568b1fd2, entries=580941, sequenceid=262487702, filesize=8.8 M 2020-11-17 11:13:43,624 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.48 MB/89630181, heapSize ~128.11 MB/134337928, currentSize=102.48 KB/104943 for 73aae5715542e1a5370f7540d2117abc in 1101ms, sequenceid=262487702, compaction requested=true 2020-11-17 11:13:51,061 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:00,352 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:04,632 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:12,001 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:18,525 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:32,304 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.23 MB heapSize=128.24 MB 2020-11-17 11:14:32,354 INFO [Thread-3525] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:33,087 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.23 MB at sequenceid=263941958 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/e7711074e8a04581bf2423524f25a814 2020-11-17 11:14:33,093 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:33,109 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e7711074e8a04581bf2423524f25a814, entries=360389, sequenceid=263941958, filesize=8.0 M 2020-11-17 11:14:33,110 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.23 MB/107200360, heapSize ~128.24 MB/134465128, currentSize=710.28 KB/727326 for e5dc5c620be19b02666a19c280ff582e in 806ms, sequenceid=263941958, compaction requested=true 2020-11-17 11:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.59 MB, max=1.83 GB, blockCount=44169, accesses=33408901, hits=28598891, hitRatio=85.60%, , cachingAccesses=28650989, cachingHits=28593091, cachingHitsRatio=99.80%, evictions=4229, evicted=0, evictedPerRun=0.0 2020-11-17 11:14:44,681 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611613698 with entries=21768, filesize=128.32 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611684653 2020-11-17 11:14:44,682 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610653378 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605610653378 2020-11-17 11:14:44,694 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610743394 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605610743394 2020-11-17 11:14:44,700 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610848082 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605610848082 2020-11-17 11:14:55,189 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.32 MB heapSize=128.27 MB 2020-11-17 11:14:55,224 INFO [Thread-3529] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:55,843 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.32 MB at sequenceid=252359960 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/4cb52f018e9d423cb25e13b8095854ac 2020-11-17 11:14:55,849 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:55,887 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/4cb52f018e9d423cb25e13b8095854ac, entries=313630, sequenceid=252359960, filesize=6.4 M 2020-11-17 11:14:55,888 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.32 MB/111487691, heapSize ~128.27 MB/134500040, currentSize=131.22 KB/134368 for 82421ce8c5c8b48c573baee83caa8305 in 699ms, sequenceid=252359960, compaction requested=true 2020-11-17 11:14:55,889 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 11:14:55,889 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/386e6a28e3be41dd839bfc11f3f92dfb, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0554c2a6f7434d01b0efd803728fb16d, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/53f3df4411644d1f9f121db051afe62f, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/4cb52f018e9d423cb25e13b8095854ac] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=762.4 M 2020-11-17 11:14:55,932 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:55,936 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:55,940 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:55,966 INFO [Thread-3531] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:57,937 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:57,939 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:58,422 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:58,674 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:58,676 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:59,449 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:59,530 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:59,743 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:59,815 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:59,887 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:14:59,964 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:15:00,591 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:15:01,030 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:15:02,915 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:15:02,917 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:15:03,622 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:15:04,797 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:15:04,992 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:15:05,928 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:15:06,003 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:15:06,469 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:15:12,067 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:15:22,348 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:15:23,532 INFO [DataStreamer for file /hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/eccdcc4ef4594a6086113240cd8e7e41] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:15:32,442 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:15:38,094 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:16:00,842 INFO [DataStreamer for file /hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/eccdcc4ef4594a6086113240cd8e7e41] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:16:09,865 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:16:10,591 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611684653 with entries=26897, filesize=128.33 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611770560 2020-11-17 11:16:33,223 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:16:37,419 INFO [DataStreamer for file /hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/eccdcc4ef4594a6086113240cd8e7e41] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:16:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 55814 ns. 2020-11-17 11:16:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 70689 ns. 2020-11-17 11:16:48,026 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:16:54,669 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=103.89 MB heapSize=128.00 MB 2020-11-17 11:16:54,730 INFO [Thread-3542] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:16:55,528 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.89 MB at sequenceid=252362539 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/53a32fe0c71b4371985927ff286e3513 2020-11-17 11:16:55,534 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:16:55,551 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/53a32fe0c71b4371985927ff286e3513, entries=346821, sequenceid=252362539, filesize=5.9 M 2020-11-17 11:16:55,553 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.89 MB/108935087, heapSize ~128.00 MB/134218064, currentSize=1000.84 KB/1024856 for 82421ce8c5c8b48c573baee83caa8305 in 884ms, sequenceid=252362539, compaction requested=false 2020-11-17 11:16:57,633 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611770560 with entries=14466, filesize=128.15 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611817606 2020-11-17 11:17:14,252 INFO [DataStreamer for file /hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/eccdcc4ef4594a6086113240cd8e7e41] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:15,797 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:35,398 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.94 MB heapSize=128.37 MB 2020-11-17 11:17:35,427 INFO [Thread-3547] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:36,295 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.94 MB at sequenceid=273357371 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/2de6837541374939a5ba881d231df050 2020-11-17 11:17:36,302 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:36,319 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/2de6837541374939a5ba881d231df050, entries=280441, sequenceid=273357371, filesize=8.4 M 2020-11-17 11:17:36,321 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~107.94 MB/113180813, heapSize ~128.37 MB/134605896, currentSize=26.76 KB/27407 for fc93f6dcc564341dcc983bf30971f73d in 923ms, sequenceid=273357371, compaction requested=true 2020-11-17 11:17:36,322 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 11:17:36,322 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/6615e712faea4e89a6ca25151bb6490e, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/5af2bb826af342c88365b621882697f2, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/158da4fded15440b9247221121998d95, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/2de6837541374939a5ba881d231df050] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=208.1 M 2020-11-17 11:17:36,371 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:36,376 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:36,382 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:36,400 INFO [Thread-3549] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 73499 ns. 2020-11-17 11:17:38,721 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:38,723 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:38,726 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:38,728 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:38,730 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:38,731 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 217286 ns. 2020-11-17 11:17:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 420800 ns. 2020-11-17 11:17:38,804 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:38,806 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:40,401 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611817606 with entries=13199, filesize=128.54 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611860374 2020-11-17 11:17:40,401 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605610942353 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605610942353 2020-11-17 11:17:40,660 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:40,663 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:42,003 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:43,332 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:43,334 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:43,518 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:44,948 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:45,085 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:45,088 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:48,387 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:48,390 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:48,392 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:56,651 INFO [DataStreamer for file /hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/eccdcc4ef4594a6086113240cd8e7e41] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:17:58,944 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 25412 ns. 2020-11-17 11:17:58,944 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18641 ns. 2020-11-17 11:18:02,581 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:14,380 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:14,382 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:15,911 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:18,461 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:18,463 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:18,465 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:18,467 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:18,574 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:18,576 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:19,124 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:19,126 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:19,240 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:19,350 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:19,461 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:20,768 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:20,892 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:20,894 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:21,004 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:21,112 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:22,564 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:22,566 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:22,568 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:22,571 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:25,640 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:25,643 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:25,646 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:27,264 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:27,372 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:29,436 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:29,510 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:29,582 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:29,585 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:30,271 INFO [DataStreamer for file /hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/edf6e45056de49069663c4c5042a806b] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:31,571 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:32,030 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.44 MB heapSize=128.03 MB 2020-11-17 11:18:32,079 INFO [Thread-3555] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:32,685 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:32,976 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.44 MB at sequenceid=263952247 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/e838aa95c5744b009270923a167ebaba 2020-11-17 11:18:33,000 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e838aa95c5744b009270923a167ebaba, entries=382958, sequenceid=263952247, filesize=7.8 M 2020-11-17 11:18:33,001 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.44 MB/105315879, heapSize ~128.03 MB/134250856, currentSize=223.25 KB/228609 for e5dc5c620be19b02666a19c280ff582e in 971ms, sequenceid=263952247, compaction requested=true 2020-11-17 11:18:34,546 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#781 average throughput is 53.01 MB/second, slept 226 time(s) and total slept time is 114415 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 11:18:34,618 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into eccdcc4ef4594a6086113240cd8e7e41(size=745.3 M), total size for store is 11.4 G. This selection was in queue for 0sec, and took 3mins, 38sec to execute. 2020-11-17 11:18:34,619 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605611695888; duration=3mins, 38sec 2020-11-17 11:18:34,619 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 11:18:34,620 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c223c73cfb1c477ba3353cdd34eb55dc, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e7711074e8a04581bf2423524f25a814, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e838aa95c5744b009270923a167ebaba] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=93.6 M 2020-11-17 11:18:34,657 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:34,674 INFO [Thread-3557] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:36,722 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:38,218 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:38,667 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:42,110 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:43,262 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:43,460 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:51,604 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:18:58,183 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#784 average throughput is 31.07 MB/second, slept 50 time(s) and total slept time is 59487 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 11:18:58,284 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into edf6e45056de49069663c4c5042a806b(size=205.0 M), total size for store is 33.5 G. This selection was in queue for 0sec, and took 1mins, 21sec to execute. 2020-11-17 11:18:58,284 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605611856321; duration=1mins, 21sec 2020-11-17 11:19:04,875 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#786 average throughput is 36.72 MB/second, slept 22 time(s) and total slept time is 19118 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 11:19:04,956 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into f43548c9958741e28a98eba5471ba5ab(size=95.2 M), total size for store is 49.8 G. This selection was in queue for 0sec, and took 30sec to execute. 2020-11-17 11:19:04,957 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605611913001; duration=30sec 2020-11-17 11:19:10,517 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:19:14,107 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:19:20,190 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:19:24,992 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:19:35,119 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=167.13 MB, max=1.83 GB, blockCount=44246, accesses=33821662, hits=28764706, hitRatio=85.05%, , cachingAccesses=28816776, cachingHits=28758796, cachingHitsRatio=99.80%, evictions=4259, evicted=0, evictedPerRun=0.0 2020-11-17 11:20:05,561 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:20:14,804 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611860374 with entries=46980, filesize=128.21 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612014766 2020-11-17 11:20:25,482 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:21:01,702 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:21:15,407 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:21:23,316 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:21:38,892 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 36673 ns. 2020-11-17 11:21:38,893 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 46159 ns. 2020-11-17 11:21:40,002 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:21:43,018 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:21:46,798 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:21:50,768 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:22:22,710 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612014766 with entries=38606, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612142674 2020-11-17 11:22:32,584 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:22:36,489 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:22:39,117 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.84 MB heapSize=128.02 MB 2020-11-17 11:22:39,178 INFO [Thread-3571] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:22:40,137 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.84 MB at sequenceid=263962548 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a226e3656b924175afd5739fde709105 2020-11-17 11:22:40,162 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a226e3656b924175afd5739fde709105, entries=406492, sequenceid=263962548, filesize=8.0 M 2020-11-17 11:22:40,165 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.84 MB/103639220, heapSize ~128.02 MB/134235032, currentSize=66.68 KB/68282 for e5dc5c620be19b02666a19c280ff582e in 1048ms, sequenceid=263962548, compaction requested=true 2020-11-17 11:22:58,946 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 11541 ns. 2020-11-17 11:22:58,946 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 34516 ns. 2020-11-17 11:23:14,795 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:23:38,741 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 202468 ns. 2020-11-17 11:23:38,742 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 54943 ns. 2020-11-17 11:23:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 615775 ns. 2020-11-17 11:23:50,349 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.88 MB heapSize=128.10 MB 2020-11-17 11:23:50,411 INFO [Thread-3576] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:23:51,163 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.88 MB at sequenceid=272752570 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/e624cf4c2b544a7a9e3cd6777991a95e 2020-11-17 11:23:51,170 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:23:51,193 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/e624cf4c2b544a7a9e3cd6777991a95e, entries=319253, sequenceid=272752570, filesize=8.6 M 2020-11-17 11:23:51,195 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.88 MB/109977513, heapSize ~128.10 MB/134327600, currentSize=582.42 KB/596401 for 0469ce862509756add4a8cdb00bc465c in 847ms, sequenceid=272752570, compaction requested=true 2020-11-17 11:23:51,196 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 11:23:51,197 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/41f7a2e6240d464299c1e372cbd3a164, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/dec2f679d7274a658ae29c17ad2369d7, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/e624cf4c2b544a7a9e3cd6777991a95e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=25.9 M 2020-11-17 11:23:51,231 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:23:51,236 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:23:51,265 INFO [Thread-3578] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:23:52,310 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:23:56,305 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#789 average throughput is 62.01 MB/second, slept 5 time(s) and total slept time is 1801 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 11:23:56,390 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 5e104d3b820a4b91acac5ddfbc858cf4(size=26.7 M), total size for store is 42.9 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 11:23:56,390 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=9, startTime=1605612231196; duration=5sec 2020-11-17 11:24:24,529 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612142674 with entries=37312, filesize=128.34 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612264489 2020-11-17 11:24:24,948 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:24:32,379 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=169.65 MB, max=1.83 GB, blockCount=44232, accesses=34545684, hits=29483544, hitRatio=85.35%, , cachingAccesses=29535723, cachingHits=29477627, cachingHitsRatio=99.80%, evictions=4289, evicted=0, evictedPerRun=0.0 2020-11-17 11:24:41,240 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:26:12,926 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=98.38 MB heapSize=128.15 MB 2020-11-17 11:26:12,969 INFO [Thread-3586] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:26:13,800 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.38 MB at sequenceid=260678489 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/5715117a6b3c49c8b85d0f41831a180d 2020-11-17 11:26:13,806 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:26:13,822 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/5715117a6b3c49c8b85d0f41831a180d, entries=414476, sequenceid=260678489, filesize=7.1 M 2020-11-17 11:26:13,824 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.38 MB/103158889, heapSize ~128.15 MB/134370648, currentSize=44.49 KB/45557 for 06f7ca9534710e37d90d9f8345864259 in 898ms, sequenceid=260678489, compaction requested=true 2020-11-17 11:26:13,825 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 11:26:13,825 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/8cf8973a5ce44d3c94d06ebbfe08eac6, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/a0461cdd999d42cfb8228bf0e2aa6f74, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/5715117a6b3c49c8b85d0f41831a180d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=66.1 M 2020-11-17 11:26:13,853 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:26:13,858 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:26:13,884 INFO [Thread-3588] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:26:16,820 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.42 MB heapSize=128.07 MB 2020-11-17 11:26:16,851 INFO [Thread-3590] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:26:17,728 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.42 MB at sequenceid=263971893 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/67697afb26304424a02ff5bb081b8433 2020-11-17 11:26:17,735 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:26:17,754 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/67697afb26304424a02ff5bb081b8433, entries=355897, sequenceid=263971893, filesize=8.2 M 2020-11-17 11:26:17,755 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.42 MB/107396030, heapSize ~128.07 MB/134287120, currentSize=1.59 MB/1668582 for e5dc5c620be19b02666a19c280ff582e in 935ms, sequenceid=263971893, compaction requested=true 2020-11-17 11:26:17,756 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 11:26:17,756 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f43548c9958741e28a98eba5471ba5ab, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a226e3656b924175afd5739fde709105, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/67697afb26304424a02ff5bb081b8433] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=111.4 M 2020-11-17 11:26:17,792 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:26:17,794 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:26:17,815 INFO [Thread-3592] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:26:21,247 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:26:30,339 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612264489 with entries=38833, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612390307 2020-11-17 11:26:39,072 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 49826 ns. 2020-11-17 11:26:39,072 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 46551 ns. 2020-11-17 11:26:39,713 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#791 average throughput is 34.69 MB/second, slept 16 time(s) and total slept time is 16705 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 11:26:39,789 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 855a454398f94ca387a6cf3df0efe9c4(size=66.4 M), total size for store is 55.6 G. This selection was in queue for 0sec, and took 25sec to execute. 2020-11-17 11:26:39,789 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605612373824; duration=25sec 2020-11-17 11:26:48,021 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:26:51,320 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#793 average throughput is 39.10 MB/second, slept 25 time(s) and total slept time is 19818 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 11:26:51,415 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into eb3d110abf044e66b3d5c7f38ac227e1(size=113.1 M), total size for store is 49.8 G. This selection was in queue for 0sec, and took 33sec to execute. 2020-11-17 11:26:51,415 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605612377755; duration=33sec 2020-11-17 11:27:33,638 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.47 MB heapSize=128.01 MB 2020-11-17 11:27:33,685 INFO [Thread-3597] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:27:34,692 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.47 MB at sequenceid=288386566 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/c25983c502c3445187b1d920c0f205b5 2020-11-17 11:27:34,698 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:27:34,714 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/c25983c502c3445187b1d920c0f205b5, entries=524408, sequenceid=288386566, filesize=9.0 M 2020-11-17 11:27:34,715 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.47 MB/94866279, heapSize ~128.01 MB/134230768, currentSize=247.40 KB/253335 for 5e5f05205d7e40b0141cd977dd009ba1 in 1077ms, sequenceid=288386566, compaction requested=true 2020-11-17 11:27:34,716 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 11:27:34,716 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/18c057a8d94840efb68cad7a07e93bba, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/d45fadb7e3354cab815c2fe38435f1de, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/c25983c502c3445187b1d920c0f205b5] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=100.8 M 2020-11-17 11:27:34,746 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:27:34,750 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:27:34,774 INFO [Thread-3599] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:27:40,833 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:27:40,835 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:27:48,511 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:27:48,512 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:27:48,639 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:27:49,964 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:27:49,967 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:27:50,484 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:27:51,676 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#795 average throughput is 56.88 MB/second, slept 14 time(s) and total slept time is 3154 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 11:27:51,765 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into 029c5040d5a541f8980c1994877a38c1(size=101.1 M), total size for store is 24.4 G. This selection was in queue for 0sec, and took 17sec to execute. 2020-11-17 11:27:51,765 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=10, startTime=1605612454715; duration=17sec 2020-11-17 11:28:07,043 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:28:09,485 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18752 ns. 2020-11-17 11:28:09,486 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 31187 ns. 2020-11-17 11:28:58,350 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612390307 with entries=44692, filesize=128.51 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612538319 2020-11-17 11:28:58,350 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611043366 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605611043366 2020-11-17 11:28:58,358 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611126488 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605611126488 2020-11-17 11:28:58,363 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611231711 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605611231711 2020-11-17 11:29:24,713 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=170.60 MB, max=1.83 GB, blockCount=44203, accesses=34731774, hits=29617970, hitRatio=85.28%, , cachingAccesses=29670124, cachingHits=29612000, cachingHitsRatio=99.80%, evictions=4319, evicted=0, evictedPerRun=0.0 2020-11-17 11:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 11:29:38,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 55917 ns. 2020-11-17 11:29:38,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 189678 ns. 2020-11-17 11:29:38,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 593496 ns. 2020-11-17 11:30:33,218 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=86.17 MB heapSize=128.00 MB 2020-11-17 11:30:33,258 INFO [Thread-3608] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:30:34,334 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=86.17 MB at sequenceid=262532693 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/9899f9f771f9459fac046a67b8577351 2020-11-17 11:30:34,340 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:30:34,357 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/9899f9f771f9459fac046a67b8577351, entries=576653, sequenceid=262532693, filesize=8.9 M 2020-11-17 11:30:34,358 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~86.17 MB/90358568, heapSize ~128.00 MB/134217800, currentSize=60.44 KB/61888 for 73aae5715542e1a5370f7540d2117abc in 1141ms, sequenceid=262532693, compaction requested=true 2020-11-17 11:30:34,359 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 11:30:34,359 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/d6069e75cd894afb80a17577bcf06d47, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/f82dfa223ecf4ee9965af68a568b1fd2, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/9899f9f771f9459fac046a67b8577351] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=81.7 M 2020-11-17 11:30:34,386 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:30:34,391 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:30:34,410 INFO [Thread-3610] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:30:41,849 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.39 MB heapSize=128.00 MB 2020-11-17 11:30:41,888 INFO [Thread-3612] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:30:42,646 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.39 MB at sequenceid=263982989 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/d763c28476fa4bad98f9e8d8b5fa3de1 2020-11-17 11:30:42,651 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:30:42,671 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d763c28476fa4bad98f9e8d8b5fa3de1, entries=355869, sequenceid=263982989, filesize=8.1 M 2020-11-17 11:30:42,672 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.39 MB/107358893, heapSize ~128.00 MB/134219344, currentSize=147.92 KB/151469 for e5dc5c620be19b02666a19c280ff582e in 823ms, sequenceid=263982989, compaction requested=true 2020-11-17 11:30:46,687 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#797 average throughput is 60.98 MB/second, slept 13 time(s) and total slept time is 1472 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 11:30:46,753 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 9f7943895cc04824a5111077215a295b(size=82.1 M), total size for store is 12.7 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 11:30:46,753 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=10, startTime=1605612634358; duration=12sec 2020-11-17 11:30:53,220 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:31:25,118 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612538319 with entries=46026, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612685076 2020-11-17 11:31:39,165 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 74657 ns. 2020-11-17 11:31:39,165 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 44475 ns. 2020-11-17 11:33:11,865 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612685076 with entries=31527, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612791798 2020-11-17 11:33:14,489 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.20 MB heapSize=128.21 MB 2020-11-17 11:33:14,510 INFO [Thread-3622] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:33:15,145 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.20 MB at sequenceid=273377486 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/d8b2283ad3d84617a606067811bbdd81 2020-11-17 11:33:15,152 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:33:15,169 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/d8b2283ad3d84617a606067811bbdd81, entries=274205, sequenceid=273377486, filesize=8.5 M 2020-11-17 11:33:15,170 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~108.20 MB/113458815, heapSize ~128.21 MB/134438336, currentSize=6.27 KB/6422 for fc93f6dcc564341dcc983bf30971f73d in 682ms, sequenceid=273377486, compaction requested=true 2020-11-17 11:33:19,619 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 11148 ns. 2020-11-17 11:33:19,619 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 29707 ns. 2020-11-17 11:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.90 MB, max=1.83 GB, blockCount=44150, accesses=34746238, hits=29620083, hitRatio=85.25%, , cachingAccesses=29672215, cachingHits=29614082, cachingHitsRatio=99.80%, evictions=4349, evicted=0, evictedPerRun=0.0 2020-11-17 11:34:46,524 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.97 MB heapSize=128.09 MB 2020-11-17 11:34:46,563 INFO [Thread-3628] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:34:47,329 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.97 MB at sequenceid=263993166 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/1e03fca7931b41e19b6e3e60a6df79ab 2020-11-17 11:34:47,337 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:34:47,356 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1e03fca7931b41e19b6e3e60a6df79ab, entries=405497, sequenceid=263993166, filesize=7.9 M 2020-11-17 11:34:47,358 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.97 MB/103778020, heapSize ~128.09 MB/134314472, currentSize=29.00 KB/29691 for e5dc5c620be19b02666a19c280ff582e in 833ms, sequenceid=263993166, compaction requested=true 2020-11-17 11:35:28,934 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612791798 with entries=41943, filesize=128.08 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612928908 2020-11-17 11:35:39,099 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 63543 ns. 2020-11-17 11:35:39,099 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 193413 ns. 2020-11-17 11:35:39,100 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 615480 ns. 2020-11-17 11:36:48,013 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:37:26,191 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.66 MB heapSize=128.01 MB 2020-11-17 11:37:26,225 INFO [Thread-3637] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:37:26,925 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.66 MB at sequenceid=272787227 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/0b1a69136efa42b1a2b39b6816ad1cb3 2020-11-17 11:37:26,932 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:37:26,948 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/0b1a69136efa42b1a2b39b6816ad1cb3, entries=321034, sequenceid=272787227, filesize=8.6 M 2020-11-17 11:37:26,949 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.66 MB/109744497, heapSize ~128.01 MB/134226352, currentSize=802.27 KB/821529 for 0469ce862509756add4a8cdb00bc465c in 758ms, sequenceid=272787227, compaction requested=true 2020-11-17 11:37:37,023 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:37:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 40017 ns. 2020-11-17 11:37:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 51229 ns. 2020-11-17 11:37:50,080 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:37:56,314 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612928908 with entries=44527, filesize=128.07 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613076252 2020-11-17 11:38:00,529 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:38:08,144 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:38:16,869 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:38:26,058 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.41 MB heapSize=128.65 MB 2020-11-17 11:38:26,087 INFO [Thread-3641] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:38:26,922 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.41 MB at sequenceid=264002345 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/8220b431fb7d434496dcfdfc224939cc 2020-11-17 11:38:26,944 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:38:26,969 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8220b431fb7d434496dcfdfc224939cc, entries=364263, sequenceid=264002345, filesize=8.1 M 2020-11-17 11:38:26,970 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.41 MB/107388735, heapSize ~128.65 MB/134899408, currentSize=30.18 KB/30902 for e5dc5c620be19b02666a19c280ff582e in 912ms, sequenceid=264002345, compaction requested=true 2020-11-17 11:38:26,971 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 11:38:26,971 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d763c28476fa4bad98f9e8d8b5fa3de1, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1e03fca7931b41e19b6e3e60a6df79ab, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8220b431fb7d434496dcfdfc224939cc] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.1 M 2020-11-17 11:38:26,993 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:38:26,996 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:38:27,020 INFO [Thread-3643] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:38:28,729 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 29623 ns. 2020-11-17 11:38:28,729 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 20344 ns. 2020-11-17 11:38:31,929 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#803 average throughput is 61.67 MB/second, slept 6 time(s) and total slept time is 1572 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 11:38:31,991 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into dd04f2bf81f0405fada8c134d77487a4(size=25.2 M), total size for store is 49.8 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 11:38:31,991 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605613106970; duration=5sec 2020-11-17 11:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.09 MB, max=1.83 GB, blockCount=44166, accesses=34895243, hits=29764140, hitRatio=85.30%, , cachingAccesses=29816284, cachingHits=29758135, cachingHitsRatio=99.80%, evictions=4379, evicted=0, evictedPerRun=0.0 2020-11-17 11:40:00,109 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613076252 with entries=37044, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613200079 2020-11-17 11:40:15,169 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:41:38,780 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 55829 ns. 2020-11-17 11:41:38,781 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 195784 ns. 2020-11-17 11:41:38,781 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 624410 ns. 2020-11-17 11:41:54,532 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613200079 with entries=34870, filesize=128.56 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613314497 2020-11-17 11:41:56,202 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.67 MB heapSize=128.53 MB 2020-11-17 11:41:56,248 INFO [Thread-3654] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:41:56,976 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.67 MB at sequenceid=264010983 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/8ca942d645244ebb9845801adf4b2c39 2020-11-17 11:41:56,982 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:41:57,001 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8ca942d645244ebb9845801adf4b2c39, entries=344781, sequenceid=264010983, filesize=8.1 M 2020-11-17 11:41:57,002 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~103.67 MB/108707931, heapSize ~128.53 MB/134773832, currentSize=672.67 KB/688816 for e5dc5c620be19b02666a19c280ff582e in 800ms, sequenceid=264010983, compaction requested=true 2020-11-17 11:42:39,042 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 52998 ns. 2020-11-17 11:42:39,042 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 32447 ns. 2020-11-17 11:43:02,218 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:43:18,073 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:43:29,687 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 9682 ns. 2020-11-17 11:43:29,687 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 30504 ns. 2020-11-17 11:44:22,493 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613314497 with entries=45180, filesize=128.21 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613462443 2020-11-17 11:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.24 MB, max=1.83 GB, blockCount=44165, accesses=35074610, hits=29943502, hitRatio=85.37%, , cachingAccesses=29995651, cachingHits=29937497, cachingHitsRatio=99.81%, evictions=4409, evicted=0, evictedPerRun=0.0 2020-11-17 11:44:54,986 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=89.15 MB heapSize=128.00 MB 2020-11-17 11:44:55,027 INFO [Thread-3662] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:44:56,155 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=89.15 MB at sequenceid=292267671 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/17c0220871464e228e925306fa5dedc0 2020-11-17 11:44:56,164 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:44:56,187 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/17c0220871464e228e925306fa5dedc0, entries=540863, sequenceid=292267671, filesize=9.0 M 2020-11-17 11:44:56,188 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~89.15 MB/93480846, heapSize ~128.00 MB/134220824, currentSize=93.59 KB/95840 for bdaa31a63b7f3d06d9909b7560333e79 in 1202ms, sequenceid=292267671, compaction requested=true 2020-11-17 11:44:59,619 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:45:15,802 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:46:08,182 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=99.71 MB heapSize=128.02 MB 2020-11-17 11:46:08,224 INFO [Thread-3666] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:46:09,065 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.71 MB at sequenceid=252399481 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/674fae529fbd4fbf9bf2488dd3bd9b9b 2020-11-17 11:46:09,071 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:46:09,088 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/674fae529fbd4fbf9bf2488dd3bd9b9b, entries=401932, sequenceid=252399481, filesize=7.3 M 2020-11-17 11:46:09,090 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~99.71 MB/104555151, heapSize ~128.02 MB/134240872, currentSize=68.53 KB/70171 for 82421ce8c5c8b48c573baee83caa8305 in 908ms, sequenceid=252399481, compaction requested=true 2020-11-17 11:46:16,960 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:46:21,624 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:46:23,007 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613462443 with entries=38131, filesize=128.31 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613582970 2020-11-17 11:46:23,007 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611310488 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605611310488 2020-11-17 11:46:23,022 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611429413 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605611429413 2020-11-17 11:46:23,028 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611499691 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605611499691 2020-11-17 11:46:23,037 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611613698 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605611613698 2020-11-17 11:46:23,042 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611684653 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605611684653 2020-11-17 11:46:23,053 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611770560 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605611770560 2020-11-17 11:46:23,063 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611817606 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605611817606 2020-11-17 11:46:23,071 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605611860374 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605611860374 2020-11-17 11:46:23,078 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612014766 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605612014766 2020-11-17 11:46:23,083 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612142674 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605612142674 2020-11-17 11:46:38,719 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.83 MB heapSize=128.10 MB 2020-11-17 11:46:38,753 INFO [Thread-3670] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:46:39,541 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.83 MB at sequenceid=264023010 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/5a2abe9330df4538b06a31dcd4ab1f04 2020-11-17 11:46:39,547 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:46:39,563 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5a2abe9330df4538b06a31dcd4ab1f04, entries=407751, sequenceid=264023010, filesize=8.0 M 2020-11-17 11:46:39,564 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~98.83 MB/103626119, heapSize ~128.10 MB/134318656, currentSize=294.80 KB/301873 for e5dc5c620be19b02666a19c280ff582e in 845ms, sequenceid=264023010, compaction requested=true 2020-11-17 11:46:39,565 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 11:46:39,565 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/dd04f2bf81f0405fada8c134d77487a4, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8ca942d645244ebb9845801adf4b2c39, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5a2abe9330df4538b06a31dcd4ab1f04] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.2 M 2020-11-17 11:46:39,589 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:46:39,592 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:46:39,614 INFO [Thread-3672] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:46:44,142 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:46:47,789 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#808 average throughput is 61.81 MB/second, slept 10 time(s) and total slept time is 3003 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 11:46:47,848 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 0f37c312690c4d20b3ca4dc8b631fd60(size=42.8 M), total size for store is 49.9 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 11:46:47,848 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605613599564; duration=8sec 2020-11-17 11:47:03,753 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.35 MB heapSize=128.01 MB 2020-11-17 11:47:03,793 INFO [Thread-3676] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:47:04,728 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.35 MB at sequenceid=262576561 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/76d6c75db21641d1892f5e6e203b2b6a 2020-11-17 11:47:04,733 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:47:04,750 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/76d6c75db21641d1892f5e6e203b2b6a, entries=583594, sequenceid=262576561, filesize=8.9 M 2020-11-17 11:47:04,752 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.35 MB/89496877, heapSize ~128.01 MB/134229592, currentSize=124.34 KB/127324 for 73aae5715542e1a5370f7540d2117abc in 999ms, sequenceid=262576561, compaction requested=true 2020-11-17 11:47:10,258 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:47:27,637 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:47:38,726 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 53026 ns. 2020-11-17 11:47:38,726 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 185725 ns. 2020-11-17 11:47:38,727 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 637975 ns. 2020-11-17 11:47:47,815 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613582970 with entries=26277, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613667796 2020-11-17 11:47:52,781 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:48:30,166 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.18 MB heapSize=128.25 MB 2020-11-17 11:48:30,203 INFO [Thread-3682] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:48:30,814 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.18 MB at sequenceid=273397263 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/f8e780aba9a04545b1b9931defa7d5bf 2020-11-17 11:48:30,821 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:48:30,841 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/f8e780aba9a04545b1b9931defa7d5bf, entries=274901, sequenceid=273397263, filesize=8.5 M 2020-11-17 11:48:30,843 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.18 MB/113433755, heapSize ~128.25 MB/134475160, currentSize=18.80 KB/19249 for fc93f6dcc564341dcc983bf30971f73d in 677ms, sequenceid=273397263, compaction requested=true 2020-11-17 11:48:38,683 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 49527 ns. 2020-11-17 11:48:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 30841 ns. 2020-11-17 11:48:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15126 ns. 2020-11-17 11:48:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 43859 ns. 2020-11-17 11:48:52,881 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.40 MB, max=1.83 GB, blockCount=44185, accesses=35088738, hits=29949383, hitRatio=85.35%, , cachingAccesses=30001554, cachingHits=29943376, cachingHitsRatio=99.81%, evictions=4439, evicted=0, evictedPerRun=0.0 2020-11-17 11:49:43,288 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=96.06 MB heapSize=128.06 MB 2020-11-17 11:49:43,328 INFO [Thread-3687] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:49:44,204 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=96.06 MB at sequenceid=260740415 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/8d693537a1cb446287178430c122426b 2020-11-17 11:49:44,209 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:49:44,234 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/8d693537a1cb446287178430c122426b, entries=435259, sequenceid=260740415, filesize=8.4 M 2020-11-17 11:49:44,236 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~96.06 MB/100726757, heapSize ~128.06 MB/134281184, currentSize=146.17 KB/149680 for 06f7ca9534710e37d90d9f8345864259 in 949ms, sequenceid=260740415, compaction requested=true 2020-11-17 11:50:01,917 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613667796 with entries=40632, filesize=128.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613801887 2020-11-17 11:50:01,918 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612264489 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605612264489 2020-11-17 11:50:19,557 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.44 MB heapSize=128.15 MB 2020-11-17 11:50:19,598 INFO [Thread-3691] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:50:20,300 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.44 MB at sequenceid=264032377 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/4f473748689c400c9be0569c49d94208 2020-11-17 11:50:20,306 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:50:20,323 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/4f473748689c400c9be0569c49d94208, entries=370815, sequenceid=264032377, filesize=8.1 M 2020-11-17 11:50:20,324 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.44 MB/106370220, heapSize ~128.15 MB/134370536, currentSize=582.71 KB/596700 for e5dc5c620be19b02666a19c280ff582e in 767ms, sequenceid=264032377, compaction requested=true 2020-11-17 11:50:37,018 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.14 MB heapSize=128.30 MB 2020-11-17 11:50:37,052 INFO [Thread-3693] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:50:37,823 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.14 MB at sequenceid=272820791 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/ef4b16df563e44d188705da6a33c9131 2020-11-17 11:50:37,829 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:50:37,846 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/ef4b16df563e44d188705da6a33c9131, entries=318399, sequenceid=272820791, filesize=8.6 M 2020-11-17 11:50:37,847 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.14 MB/110248240, heapSize ~128.30 MB/134534288, currentSize=819.58 KB/839245 for 0469ce862509756add4a8cdb00bc465c in 829ms, sequenceid=272820791, compaction requested=true 2020-11-17 11:50:37,848 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 11:50:37,848 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/5e104d3b820a4b91acac5ddfbc858cf4, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/0b1a69136efa42b1a2b39b6816ad1cb3, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/ef4b16df563e44d188705da6a33c9131] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=44.0 M 2020-11-17 11:50:37,874 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:50:37,876 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:50:37,893 INFO [Thread-3695] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:50:42,660 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:50:45,894 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#814 average throughput is 65.24 MB/second, slept 9 time(s) and total slept time is 2959 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 11:50:45,981 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 11a6750bab1447e786d6224550094cb9(size=44.8 M), total size for store is 43.0 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 11:50:45,981 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=9, startTime=1605613837848; duration=8sec 2020-11-17 11:51:15,706 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:51:28,919 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613801887 with entries=27065, filesize=128.27 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613888884 2020-11-17 11:51:52,732 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:51:59,522 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:01,404 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:06,081 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:09,246 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:12,970 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:12,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:24,477 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.62 MB heapSize=128.00 MB 2020-11-17 11:52:24,524 INFO [Thread-3703] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:25,446 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.63 MB at sequenceid=288455087 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/9ae8fc0011554da99469f17c8e785402 2020-11-17 11:52:25,472 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/9ae8fc0011554da99469f17c8e785402, entries=521671, sequenceid=288455087, filesize=9.1 M 2020-11-17 11:52:25,474 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.63 MB/95029759, heapSize ~128.01 MB/134228264, currentSize=40.64 KB/41617 for 5e5f05205d7e40b0141cd977dd009ba1 in 997ms, sequenceid=288455087, compaction requested=true 2020-11-17 11:52:25,475 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 11:52:25,475 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/ad18bc9861a24f818af5160b588ec702, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/029c5040d5a541f8980c1994877a38c1, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/9ae8fc0011554da99469f17c8e785402] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=231.7 M 2020-11-17 11:52:25,517 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:25,525 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:25,549 INFO [Thread-3705] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:29,106 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:31,342 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:34,383 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:34,490 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:38,162 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:38,169 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:38,174 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:38,176 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:38,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:44,992 INFO [DataStreamer for file /hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/81342393f44c40f4b98237a24b5f102c] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:51,023 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:51,025 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:53,060 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:53,710 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:53,713 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:54,655 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:54,796 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:55,034 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:55,036 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:55,038 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:56,920 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:57,602 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:58,473 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:58,475 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:58,477 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:58,478 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:58,854 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:52:59,116 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:53:01,783 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#816 average throughput is 58.39 MB/second, slept 33 time(s) and total slept time is 8483 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 11:53:01,875 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into 81342393f44c40f4b98237a24b5f102c(size=229.3 M), total size for store is 24.4 G. This selection was in queue for 0sec, and took 36sec to execute. 2020-11-17 11:53:01,876 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=11, startTime=1605613945475; duration=36sec 2020-11-17 11:53:10,971 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:53:18,085 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:53:18,642 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613888884 with entries=33593, filesize=128.24 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613998596 2020-11-17 11:53:18,642 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612390307 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605612390307 2020-11-17 11:53:18,654 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612538319 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605612538319 2020-11-17 11:53:18,664 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612685076 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605612685076 2020-11-17 11:53:18,668 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612791798 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605612791798 2020-11-17 11:53:18,673 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605612928908 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605612928908 2020-11-17 11:53:18,682 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613076252 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605613076252 2020-11-17 11:53:18,687 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613200079 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605613200079 2020-11-17 11:53:18,692 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613314497 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605613314497 2020-11-17 11:53:22,474 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:53:30,962 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:53:36,499 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:53:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 35209 ns. 2020-11-17 11:53:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 17452 ns. 2020-11-17 11:53:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 28191 ns. 2020-11-17 11:53:38,750 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 442801 ns. 2020-11-17 11:53:38,750 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 58242 ns. 2020-11-17 11:53:38,750 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 53639 ns. 2020-11-17 11:53:38,750 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 167400 ns. 2020-11-17 11:53:39,606 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:53:45,101 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:53:55,876 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:00,249 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:07,854 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.76 MB heapSize=128.04 MB 2020-11-17 11:54:07,898 INFO [Thread-3713] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:08,649 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.76 MB at sequenceid=252410044 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/5312243083124e09ba1d98709d07985e 2020-11-17 11:54:08,661 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:08,681 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/5312243083124e09ba1d98709d07985e, entries=318070, sequenceid=252410044, filesize=6.6 M 2020-11-17 11:54:08,683 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.76 MB/110898071, heapSize ~128.04 MB/134261544, currentSize=311.96 KB/319446 for 82421ce8c5c8b48c573baee83caa8305 in 829ms, sequenceid=252410044, compaction requested=true 2020-11-17 11:54:08,684 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 11:54:08,684 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/53a32fe0c71b4371985927ff286e3513, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/674fae529fbd4fbf9bf2488dd3bd9b9b, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/5312243083124e09ba1d98709d07985e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=19.7 M 2020-11-17 11:54:08,728 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:08,741 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:08,778 INFO [Thread-3715] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:08,926 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:13,794 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#818 average throughput is 61.38 MB/second, slept 6 time(s) and total slept time is 1536 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 11:54:13,862 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 5a8dabbb04f94078aa06c22497430bbb(size=19.7 M), total size for store is 11.4 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 11:54:13,862 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605614048683; duration=5sec 2020-11-17 11:54:18,309 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.86 MB heapSize=128.16 MB 2020-11-17 11:54:18,451 INFO [Thread-3717] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:19,454 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.86 MB at sequenceid=264042345 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/95242f2e172a423c805cde45cc2ea6c9 2020-11-17 11:54:19,462 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:19,480 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/95242f2e172a423c805cde45cc2ea6c9, entries=364843, sequenceid=264042345, filesize=8.0 M 2020-11-17 11:54:19,481 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.86 MB/106806788, heapSize ~128.16 MB/134380568, currentSize=1.95 MB/2048937 for e5dc5c620be19b02666a19c280ff582e in 1172ms, sequenceid=264042345, compaction requested=true 2020-11-17 11:54:19,481 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 11:54:19,482 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/0f37c312690c4d20b3ca4dc8b631fd60, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/4f473748689c400c9be0569c49d94208, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/95242f2e172a423c805cde45cc2ea6c9] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=58.9 M 2020-11-17 11:54:19,517 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:19,520 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:19,557 INFO [Thread-3719] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:24,515 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:31,169 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#820 average throughput is 60.98 MB/second, slept 13 time(s) and total slept time is 3367 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 11:54:31,237 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:31,262 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into cd12e087cf9c43fbad11702c976783e3(size=60.3 M), total size for store is 49.9 G. This selection was in queue for 0sec, and took 11sec to execute. 2020-11-17 11:54:31,262 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605614059481; duration=11sec 2020-11-17 11:54:33,281 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613998596 with entries=22394, filesize=128.47 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614073252 2020-11-17 11:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=169.03 MB, max=1.83 GB, blockCount=44259, accesses=35807731, hits=30608539, hitRatio=85.48%, , cachingAccesses=30660777, cachingHits=30602472, cachingHitsRatio=99.81%, evictions=4469, evicted=0, evictedPerRun=0.0 2020-11-17 11:54:42,344 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:54:47,542 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:55:10,455 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:55:20,819 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:55:28,094 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:55:41,318 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:55:51,190 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:56:26,274 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:56:32,595 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614073252 with entries=36476, filesize=128.59 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614192560 2020-11-17 11:57:04,418 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:57:58,644 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:58:33,842 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:58:38,818 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 46954 ns. 2020-11-17 11:58:45,854 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.66 MB heapSize=128.13 MB 2020-11-17 11:58:45,898 INFO [Thread-3731] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:58:46,666 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.66 MB at sequenceid=264053527 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/750ca2d6b9c44ee2847a5bc245969718 2020-11-17 11:58:46,673 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 11:58:46,692 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/750ca2d6b9c44ee2847a5bc245969718, entries=382111, sequenceid=264053527, filesize=7.9 M 2020-11-17 11:58:46,694 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.66 MB/105549615, heapSize ~128.13 MB/134353624, currentSize=371.36 KB/380273 for e5dc5c620be19b02666a19c280ff582e in 840ms, sequenceid=264053527, compaction requested=true 2020-11-17 11:58:48,735 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 35463 ns. 2020-11-17 11:58:48,735 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 68502 ns. 2020-11-17 11:59:08,797 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614192560 with entries=46818, filesize=128.09 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614348762 2020-11-17 11:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=169.32 MB, max=1.83 GB, blockCount=44250, accesses=36176055, hits=30976841, hitRatio=85.63%, , cachingAccesses=31029101, cachingHits=30970774, cachingHitsRatio=99.81%, evictions=4499, evicted=0, evictedPerRun=0.0 2020-11-17 11:59:39,030 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 231573 ns. 2020-11-17 11:59:39,032 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 733702 ns. 2020-11-17 11:59:39,032 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 79881 ns. 2020-11-17 11:59:39,032 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 54620 ns. 2020-11-17 12:01:15,655 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614348762 with entries=40284, filesize=128.57 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614475621 2020-11-17 12:02:03,369 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:02:06,981 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:02:25,178 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.24 MB heapSize=128.00 MB 2020-11-17 12:02:25,216 INFO [Thread-3743] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:02:25,977 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.24 MB at sequenceid=264062940 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/cea553d48ffe474cbbfc00fed9c7a1b1 2020-11-17 12:02:25,983 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:02:26,001 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/cea553d48ffe474cbbfc00fed9c7a1b1, entries=371597, sequenceid=264062940, filesize=8.1 M 2020-11-17 12:02:26,002 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.24 MB/106160579, heapSize ~128.00 MB/134218776, currentSize=21.03 KB/21538 for e5dc5c620be19b02666a19c280ff582e in 825ms, sequenceid=264062940, compaction requested=true 2020-11-17 12:02:26,004 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 12:02:26,004 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/cd12e087cf9c43fbad11702c976783e3, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/750ca2d6b9c44ee2847a5bc245969718, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/cea553d48ffe474cbbfc00fed9c7a1b1] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=76.3 M 2020-11-17 12:02:26,029 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:02:26,032 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:02:26,057 INFO [Thread-3745] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:02:34,983 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:02:40,701 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#823 average throughput is 62.17 MB/second, slept 18 time(s) and total slept time is 5156 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 12:02:40,769 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into ea423fc5664942e4976ee5dc0517f876(size=78.0 M), total size for store is 49.9 G. This selection was in queue for 0sec, and took 14sec to execute. 2020-11-17 12:02:40,769 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605614546002; duration=14sec 2020-11-17 12:02:44,580 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.84 MB heapSize=128.00 MB 2020-11-17 12:02:44,607 INFO [Thread-3748] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:02:45,694 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.84 MB at sequenceid=262618181 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/1ee13bb23ea741e7aa01e005a7b6c921 2020-11-17 12:02:45,699 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:02:45,715 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/1ee13bb23ea741e7aa01e005a7b6c921, entries=576399, sequenceid=262618181, filesize=8.8 M 2020-11-17 12:02:45,717 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.84 MB/90005696, heapSize ~128.00 MB/134219384, currentSize=252.25 KB/258308 for 73aae5715542e1a5370f7540d2117abc in 1137ms, sequenceid=262618181, compaction requested=true 2020-11-17 12:02:45,717 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 12:02:45,718 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/9f7943895cc04824a5111077215a295b, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/76d6c75db21641d1892f5e6e203b2b6a, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/1ee13bb23ea741e7aa01e005a7b6c921] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=99.8 M 2020-11-17 12:02:45,749 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:02:45,753 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:02:45,784 INFO [Thread-3750] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:02:57,501 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:02:57,504 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:03:00,652 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#825 average throughput is 61.30 MB/second, slept 10 time(s) and total slept time is 1489 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 12:03:00,734 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 6b2af9858550412cbe967673d28f83c0(size=100.3 M), total size for store is 12.7 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 12:03:00,734 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=10, startTime=1605614565717; duration=15sec 2020-11-17 12:03:02,231 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614475621 with entries=31997, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614582210 2020-11-17 12:03:06,385 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:03:22,339 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:03:29,654 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:03:39,002 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 45664 ns. 2020-11-17 12:03:42,602 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:03:49,520 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14995 ns. 2020-11-17 12:03:49,520 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 27830 ns. 2020-11-17 12:03:59,543 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.13 MB heapSize=128.25 MB 2020-11-17 12:03:59,583 INFO [Thread-3755] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:04:00,225 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.13 MB at sequenceid=273417405 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/698e0dc141e249c7ab376f41dc002b49 2020-11-17 12:04:00,230 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:04:00,252 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/698e0dc141e249c7ab376f41dc002b49, entries=275915, sequenceid=273417405, filesize=8.5 M 2020-11-17 12:04:00,253 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.13 MB/113385942, heapSize ~128.25 MB/134484656, currentSize=18.43 KB/18876 for fc93f6dcc564341dcc983bf30971f73d in 710ms, sequenceid=273417405, compaction requested=true 2020-11-17 12:04:00,254 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 12:04:00,254 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/d8b2283ad3d84617a606067811bbdd81, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/f8e780aba9a04545b1b9931defa7d5bf, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/698e0dc141e249c7ab376f41dc002b49] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=25.4 M 2020-11-17 12:04:00,280 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:04:00,285 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:04:00,305 INFO [Thread-3757] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:04:05,802 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#827 average throughput is 58.85 MB/second, slept 6 time(s) and total slept time is 2884 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 12:04:05,868 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into c3c28bdb02fe4e8c835f6e08a468f7e7(size=26.1 M), total size for store is 33.6 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 12:04:05,868 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605614640254; duration=5sec 2020-11-17 12:04:11,552 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.77 MB heapSize=128.00 MB 2020-11-17 12:04:11,576 INFO [Thread-3759] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:04:12,251 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.77 MB at sequenceid=272854930 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/683c063688b64bfa81b644ad43790d2b 2020-11-17 12:04:12,258 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:04:12,274 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/683c063688b64bfa81b644ad43790d2b, entries=319447, sequenceid=272854930, filesize=8.6 M 2020-11-17 12:04:12,281 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.77 MB/109856085, heapSize ~128.00 MB/134220256, currentSize=64.06 KB/65601 for 0469ce862509756add4a8cdb00bc465c in 729ms, sequenceid=272854930, compaction requested=true 2020-11-17 12:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=170.53 MB, max=1.83 GB, blockCount=44217, accesses=36220788, hits=30986485, hitRatio=85.55%, , cachingAccesses=31038718, cachingHits=30980374, cachingHitsRatio=99.81%, evictions=4529, evicted=0, evictedPerRun=0.0 2020-11-17 12:04:40,358 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:05:12,118 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614582210 with entries=39623, filesize=128.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614712071 2020-11-17 12:05:12,360 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:05:38,871 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 832554 ns. 2020-11-17 12:05:38,871 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 51564 ns. 2020-11-17 12:05:38,872 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 246797 ns. 2020-11-17 12:05:38,872 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 73545 ns. 2020-11-17 12:05:57,107 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.81 MB heapSize=128.23 MB 2020-11-17 12:05:57,147 INFO [Thread-3766] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:05:57,861 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.81 MB at sequenceid=264071814 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/0b39457029144923ad5b7451a628c527 2020-11-17 12:05:57,866 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:05:57,884 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/0b39457029144923ad5b7451a628c527, entries=352475, sequenceid=264071814, filesize=8.2 M 2020-11-17 12:05:57,891 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.81 MB/107808973, heapSize ~128.23 MB/134461056, currentSize=25.76 KB/26374 for e5dc5c620be19b02666a19c280ff582e in 784ms, sequenceid=264071814, compaction requested=true 2020-11-17 12:06:24,478 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=98.88 MB heapSize=128.02 MB 2020-11-17 12:06:24,530 INFO [Thread-3769] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:06:25,402 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.88 MB at sequenceid=260784304 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/1a353ffe044a496fbf0aeef6dbb5965b 2020-11-17 12:06:25,407 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:06:25,423 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/1a353ffe044a496fbf0aeef6dbb5965b, entries=401533, sequenceid=260784304, filesize=7.7 M 2020-11-17 12:06:25,424 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.88 MB/103680775, heapSize ~128.02 MB/134241232, currentSize=114.53 KB/117278 for 06f7ca9534710e37d90d9f8345864259 in 946ms, sequenceid=260784304, compaction requested=true 2020-11-17 12:06:25,424 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 12:06:25,425 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/855a454398f94ca387a6cf3df0efe9c4, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/8d693537a1cb446287178430c122426b, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/1a353ffe044a496fbf0aeef6dbb5965b] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=82.5 M 2020-11-17 12:06:25,452 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:06:25,457 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:06:25,479 INFO [Thread-3771] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:06:31,710 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614712071 with entries=24437, filesize=128.16 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614791690 2020-11-17 12:06:43,071 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#831 average throughput is 61.89 MB/second, slept 19 time(s) and total slept time is 6332 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 12:06:43,159 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 1737f9ae6d994c2aad724c84852f2b89(size=82.9 M), total size for store is 55.6 G. This selection was in queue for 0sec, and took 17sec to execute. 2020-11-17 12:06:43,159 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605614785424; duration=17sec 2020-11-17 12:06:48,014 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:06:55,445 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.20 MB heapSize=128.02 MB 2020-11-17 12:06:55,486 INFO [Thread-3776] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:06:56,071 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.20 MB at sequenceid=252426446 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/d044baed1a254fafa87e87395da13bd6 2020-11-17 12:06:56,076 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:06:56,093 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d044baed1a254fafa87e87395da13bd6, entries=325478, sequenceid=252426446, filesize=6.8 M 2020-11-17 12:06:56,094 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.20 MB/110308759, heapSize ~128.02 MB/134243856, currentSize=19.51 KB/19976 for 82421ce8c5c8b48c573baee83caa8305 in 649ms, sequenceid=252426446, compaction requested=true 2020-11-17 12:08:04,214 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614791690 with entries=28320, filesize=128.19 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614884184 2020-11-17 12:08:48,061 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:08:51,426 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:08:58,750 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 27928 ns. 2020-11-17 12:08:58,751 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18564 ns. 2020-11-17 12:09:18,369 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614884184 with entries=23321, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614958342 2020-11-17 12:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=170.53 MB, max=1.83 GB, blockCount=44213, accesses=36383109, hits=31131053, hitRatio=85.56%, , cachingAccesses=31183299, cachingHits=31124922, cachingHitsRatio=99.81%, evictions=4559, evicted=0, evictedPerRun=0.0 2020-11-17 12:09:39,278 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 50901 ns. 2020-11-17 12:10:39,590 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 86729 ns. 2020-11-17 12:10:46,181 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.02 MB heapSize=128.02 MB 2020-11-17 12:10:46,221 INFO [Thread-3787] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:10:46,969 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.02 MB at sequenceid=264084016 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/7a0b496cb4df4b47a28a4cc708c911c2 2020-11-17 12:10:46,975 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:10:46,992 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7a0b496cb4df4b47a28a4cc708c911c2, entries=403477, sequenceid=264084016, filesize=7.9 M 2020-11-17 12:10:46,993 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.02 MB/103827062, heapSize ~128.02 MB/134235480, currentSize=1.31 MB/1374695 for e5dc5c620be19b02666a19c280ff582e in 812ms, sequenceid=264084016, compaction requested=true 2020-11-17 12:10:46,994 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 12:10:46,994 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ea423fc5664942e4976ee5dc0517f876, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/0b39457029144923ad5b7451a628c527, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7a0b496cb4df4b47a28a4cc708c911c2] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=94.2 M 2020-11-17 12:10:47,018 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:10:47,021 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:10:47,042 INFO [Thread-3789] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:10:59,296 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:11:05,331 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#834 average throughput is 60.82 MB/second, slept 20 time(s) and total slept time is 5517 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 12:11:05,457 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 5d19a10052d342f1973bf3290751fd92(size=95.8 M), total size for store is 49.9 G. This selection was in queue for 0sec, and took 18sec to execute. 2020-11-17 12:11:05,457 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605615046993; duration=18sec 2020-11-17 12:11:05,733 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614958342 with entries=32462, filesize=128.20 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615065692 2020-11-17 12:11:09,207 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:11:38,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 92010 ns. 2020-11-17 12:11:38,715 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 225266 ns. 2020-11-17 12:11:38,715 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 806094 ns. 2020-11-17 12:11:46,503 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.57 MB heapSize=128.14 MB 2020-11-17 12:11:46,536 INFO [Thread-3794] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:11:47,196 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.57 MB at sequenceid=252432678 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/6436b2e4aa6c4f49a9e4d3986b0c8a8f 2020-11-17 12:11:47,203 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:11:47,207 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:11:47,226 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/6436b2e4aa6c4f49a9e4d3986b0c8a8f, entries=308627, sequenceid=252432678, filesize=6.4 M 2020-11-17 12:11:47,234 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.57 MB/111746289, heapSize ~128.14 MB/134364680, currentSize=249.38 KB/255370 for 82421ce8c5c8b48c573baee83caa8305 in 730ms, sequenceid=252432678, compaction requested=true 2020-11-17 12:11:47,238 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 12:11:47,238 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/5a8dabbb04f94078aa06c22497430bbb, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d044baed1a254fafa87e87395da13bd6, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/6436b2e4aa6c4f49a9e4d3986b0c8a8f] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=32.8 M 2020-11-17 12:11:47,276 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:11:47,278 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:11:47,302 INFO [Thread-3796] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:11:55,949 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#836 average throughput is 60.14 MB/second, slept 9 time(s) and total slept time is 2937 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 12:11:56,044 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 8340445874f44ee285f39a95e1d324ba(size=32.8 M), total size for store is 11.4 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 12:11:56,044 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605615107234; duration=8sec 2020-11-17 12:12:14,648 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615065692 with entries=21600, filesize=128.55 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615134617 2020-11-17 12:13:17,830 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:13:23,977 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:13:31,887 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:13:41,031 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:13:50,532 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615134617 with entries=29311, filesize=128.33 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615230492 2020-11-17 12:13:59,837 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:14:08,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 25424 ns. 2020-11-17 12:14:08,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15791 ns. 2020-11-17 12:14:13,110 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:14:18,271 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:14:18,360 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.51 MB heapSize=128.14 MB 2020-11-17 12:14:18,412 INFO [Thread-3805] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:14:19,206 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.51 MB at sequenceid=264093139 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/96e3466f38f443b0a4255b1f59564990 2020-11-17 12:14:19,227 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/96e3466f38f443b0a4255b1f59564990, entries=397826, sequenceid=264093139, filesize=8.0 M 2020-11-17 12:14:19,228 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~99.51 MB/104347111, heapSize ~128.14 MB/134365704, currentSize=484.83 KB/496471 for e5dc5c620be19b02666a19c280ff582e in 868ms, sequenceid=264093139, compaction requested=true 2020-11-17 12:14:19,229 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 12:14:19,229 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/eb3d110abf044e66b3d5c7f38ac227e1, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5d19a10052d342f1973bf3290751fd92, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/96e3466f38f443b0a4255b1f59564990] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=216.8 M 2020-11-17 12:14:19,258 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:14:19,265 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:14:19,288 INFO [Thread-3807] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:14:28,132 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=170.57 MB, max=1.83 GB, blockCount=44218, accesses=36563433, hits=31268028, hitRatio=85.52%, , cachingAccesses=31320282, cachingHits=31261884, cachingHitsRatio=99.81%, evictions=4589, evicted=0, evictedPerRun=0.0 2020-11-17 12:14:43,509 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/3183b857c28f4c2badb7959ed3ec1687] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:15:01,198 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#838 average throughput is 60.19 MB/second, slept 45 time(s) and total slept time is 15431 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 12:15:01,272 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 3183b857c28f4c2badb7959ed3ec1687(size=220.1 M), total size for store is 49.9 G. This selection was in queue for 0sec, and took 42sec to execute. 2020-11-17 12:15:01,272 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605615259228; duration=42sec 2020-11-17 12:15:09,677 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615230492 with entries=24148, filesize=128.32 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615309648 2020-11-17 12:15:14,583 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:15:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 45018 ns. 2020-11-17 12:15:43,094 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.65 MB heapSize=128.00 MB 2020-11-17 12:15:43,118 INFO [Thread-3814] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:15:44,072 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.65 MB at sequenceid=288519453 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/1301cf27422041ab95866faa3d14f2c9 2020-11-17 12:15:44,077 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:15:44,094 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/1301cf27422041ab95866faa3d14f2c9, entries=521778, sequenceid=288519453, filesize=9.0 M 2020-11-17 12:15:44,095 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.65 MB/95050005, heapSize ~128.00 MB/134219864, currentSize=58.14 KB/59532 for 5e5f05205d7e40b0141cd977dd009ba1 in 1002ms, sequenceid=288519453, compaction requested=true 2020-11-17 12:16:10,783 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:16:21,186 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.61 MB heapSize=128.18 MB 2020-11-17 12:16:21,225 INFO [Thread-3818] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:16:21,858 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.61 MB at sequenceid=252438610 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/be212332b7e44537add2c5912b98d0bc 2020-11-17 12:16:21,865 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:16:21,888 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/be212332b7e44537add2c5912b98d0bc, entries=308712, sequenceid=252438610, filesize=6.3 M 2020-11-17 12:16:21,890 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.61 MB/111783640, heapSize ~128.18 MB/134408664, currentSize=21.76 KB/22282 for 82421ce8c5c8b48c573baee83caa8305 in 704ms, sequenceid=252438610, compaction requested=true 2020-11-17 12:16:39,076 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 72287 ns. 2020-11-17 12:16:45,591 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615309648 with entries=30724, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615405568 2020-11-17 12:16:46,234 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:16:49,260 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.60 MB heapSize=128.00 MB 2020-11-17 12:16:49,291 INFO [Thread-3822] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:16:50,322 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.60 MB at sequenceid=292353242 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/b8cf0263bebc4109918676a140e8e9f5 2020-11-17 12:16:50,343 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/b8cf0263bebc4109918676a140e8e9f5, entries=547782, sequenceid=292353242, filesize=8.8 M 2020-11-17 12:16:50,345 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~88.60 MB/92899500, heapSize ~128.00 MB/134218312, currentSize=53.67 KB/54962 for bdaa31a63b7f3d06d9909b7560333e79 in 1084ms, sequenceid=292353242, compaction requested=true 2020-11-17 12:16:50,345 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-17 12:16:50,345 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/535592258538419fb7322a690da9cde3, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/17c0220871464e228e925306fa5dedc0, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/b8cf0263bebc4109918676a140e8e9f5] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp, totalSize=44.6 M 2020-11-17 12:16:50,374 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:16:50,378 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:16:50,399 INFO [Thread-3824] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:16:50,794 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.54 MB heapSize=128.02 MB 2020-11-17 12:16:50,849 INFO [Thread-3826] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:16:51,592 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.54 MB at sequenceid=260811890 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/bd5f05c2d48a4e8d8bbb2cef14ec3355 2020-11-17 12:16:51,597 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:16:51,612 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/bd5f05c2d48a4e8d8bbb2cef14ec3355, entries=370479, sequenceid=260811890, filesize=7.0 M 2020-11-17 12:16:51,613 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.54 MB/106470552, heapSize ~128.02 MB/134239560, currentSize=102.29 KB/104749 for 06f7ca9534710e37d90d9f8345864259 in 819ms, sequenceid=260811890, compaction requested=true 2020-11-17 12:16:55,788 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:16:55,791 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:16:55,857 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:16:56,440 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:16:57,515 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: bdaa31a63b7f3d06d9909b7560333e79#e#compaction#842 average throughput is 60.66 MB/second, slept 5 time(s) and total slept time is 988 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 12:16:57,578 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of bdaa31a63b7f3d06d9909b7560333e79 into 96397fb03fa743119f6c3bd7df9e6033(size=44.5 M), total size for store is 31.9 G. This selection was in queue for 0sec, and took 7sec to execute. 2020-11-17 12:16:57,578 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., storeName=e, priority=11, startTime=1605615410345; duration=7sec 2020-11-17 12:17:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 65715 ns. 2020-11-17 12:17:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 210395 ns. 2020-11-17 12:17:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 829290 ns. 2020-11-17 12:17:40,135 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.28 MB heapSize=128.63 MB 2020-11-17 12:17:40,157 INFO [Thread-3829] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:17:40,831 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.28 MB at sequenceid=272889179 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/77e3fa204f4d40efaba498f4b4aacac0 2020-11-17 12:17:40,836 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:17:40,936 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/77e3fa204f4d40efaba498f4b4aacac0, entries=320828, sequenceid=272889179, filesize=8.7 M 2020-11-17 12:17:40,941 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.28 MB/110397464, heapSize ~128.63 MB/134875808, currentSize=601.21 KB/615640 for 0469ce862509756add4a8cdb00bc465c in 806ms, sequenceid=272889179, compaction requested=true 2020-11-17 12:17:40,942 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 12:17:40,942 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/11a6750bab1447e786d6224550094cb9, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/683c063688b64bfa81b644ad43790d2b, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/77e3fa204f4d40efaba498f4b4aacac0] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=62.1 M 2020-11-17 12:17:40,971 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:17:40,975 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:17:40,993 INFO [Thread-3831] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:17:41,174 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.17 MB heapSize=128.48 MB 2020-11-17 12:17:41,211 INFO [Thread-3833] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:17:41,992 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.17 MB at sequenceid=264101819 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/21f8ccc3d6a743d8b24ddf8876d6b9a9 2020-11-17 12:17:41,997 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:17:42,023 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/21f8ccc3d6a743d8b24ddf8876d6b9a9, entries=364597, sequenceid=264101819, filesize=8.1 M 2020-11-17 12:17:42,025 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.17 MB/107137006, heapSize ~128.48 MB/134716928, currentSize=618.34 KB/633182 for e5dc5c620be19b02666a19c280ff582e in 851ms, sequenceid=264101819, compaction requested=true 2020-11-17 12:17:49,260 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:17:52,154 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:17:53,145 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#845 average throughput is 60.09 MB/second, slept 12 time(s) and total slept time is 3560 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 12:17:53,231 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 95e0cc948570482cacba08ca7eb15343(size=62.9 M), total size for store is 43.0 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 12:17:53,231 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=9, startTime=1605615460941; duration=12sec 2020-11-17 12:18:01,209 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615405568 with entries=21823, filesize=128.61 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615481188 2020-11-17 12:18:01,209 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613462443 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605613462443 2020-11-17 12:18:01,220 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613582970 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605613582970 2020-11-17 12:18:01,226 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613667796 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605613667796 2020-11-17 12:18:01,230 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613801887 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605613801887 2020-11-17 12:18:01,236 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613888884 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605613888884 2020-11-17 12:18:01,241 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605613998596 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605613998596 2020-11-17 12:18:01,246 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614073252 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605614073252 2020-11-17 12:18:01,250 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614192560 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605614192560 2020-11-17 12:18:01,255 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614348762 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605614348762 2020-11-17 12:18:01,405 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:10,848 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.55 MB heapSize=128.08 MB 2020-11-17 12:18:10,874 INFO [Thread-3836] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:11,951 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.55 MB at sequenceid=262659204 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/9c7720b798914af4a1b435d939dba4aa 2020-11-17 12:18:11,957 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:11,977 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/9c7720b798914af4a1b435d939dba4aa, entries=581318, sequenceid=262659204, filesize=8.8 M 2020-11-17 12:18:11,980 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.55 MB/89705542, heapSize ~128.08 MB/134304848, currentSize=150.32 KB/153926 for 73aae5715542e1a5370f7540d2117abc in 1132ms, sequenceid=262659204, compaction requested=true 2020-11-17 12:18:11,981 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 12:18:11,981 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/1b4769138044494cb0e1b79523dd0948, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/4085aa36dee1496ebd755a1c78008085, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/6b2af9858550412cbe967673d28f83c0, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/9c7720b798914af4a1b435d939dba4aa] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=712.0 M 2020-11-17 12:18:12,046 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:12,050 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:12,059 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:12,081 INFO [Thread-3838] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:13,978 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:14,056 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:14,553 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:26,222 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:26,224 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:26,846 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:31,846 INFO [DataStreamer for file /hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/bfb26c7f7634492e8fd28f8bd6bbde1f] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:40,058 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:50,787 INFO [DataStreamer for file /hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/bfb26c7f7634492e8fd28f8bd6bbde1f] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:51,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:18:55,791 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:04,251 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:04,253 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:04,255 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:04,561 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:07,133 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:07,135 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:07,220 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:08,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14753 ns. 2020-11-17 12:19:08,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 47798 ns. 2020-11-17 12:19:11,028 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:11,256 INFO [DataStreamer for file /hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/bfb26c7f7634492e8fd28f8bd6bbde1f] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:15,039 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:15,379 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:15,858 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:15,861 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:17,662 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:20,731 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:20,741 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:20,856 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:20,859 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:21,019 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:22,645 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:24,536 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.71 MB heapSize=128.00 MB 2020-11-17 12:19:24,564 INFO [Thread-3848] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:25,225 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:25,227 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:25,265 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.71 MB at sequenceid=273437571 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/6864873a0593460489ad34c51b883dcf 2020-11-17 12:19:25,271 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:25,285 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/6864873a0593460489ad34c51b883dcf, entries=278497, sequenceid=273437571, filesize=8.4 M 2020-11-17 12:19:25,286 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~107.71 MB/112939719, heapSize ~128.00 MB/134219736, currentSize=13.41 KB/13727 for fc93f6dcc564341dcc983bf30971f73d in 750ms, sequenceid=273437571, compaction requested=true 2020-11-17 12:19:25,976 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:26,100 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:26,784 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:26,882 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:27,336 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:27,845 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:28,316 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:28,420 INFO [DataStreamer for file /hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/bfb26c7f7634492e8fd28f8bd6bbde1f] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:28,700 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:29,421 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:29,424 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:31,297 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:31,299 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:31,595 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:31,598 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:32,549 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:32,551 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:32,553 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:32,555 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:32,557 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:32,559 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:32,679 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:32,806 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:33,276 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:33,389 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:33,391 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:33,505 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:33,618 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:34,171 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:34,311 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:34,445 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:34,918 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:35,033 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:35,156 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:35,230 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:35,233 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:35,922 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:35,925 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=169.77 MB, max=1.83 GB, blockCount=44192, accesses=36691105, hits=31274522, hitRatio=85.24%, , cachingAccesses=31326522, cachingHits=31268084, cachingHitsRatio=99.81%, evictions=4619, evicted=0, evictedPerRun=0.0 2020-11-17 12:19:36,045 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:36,132 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:36,134 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:36,218 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:36,221 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:36,222 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:39,424 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:39,428 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:39,990 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:39,992 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:40,208 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:40,483 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:40,675 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:41,316 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:41,318 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:41,469 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:42,160 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:42,298 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:42,300 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:44,229 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:44,231 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:44,233 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:44,237 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:45,147 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:45,257 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:45,360 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:45,818 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:45,821 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:45,894 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:45,896 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:45,974 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:45,975 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:49,029 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615481188 with entries=32899, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615588989 2020-11-17 12:19:49,029 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614475621 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605614475621 2020-11-17 12:19:49,046 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614582210 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605614582210 2020-11-17 12:19:49,052 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614712071 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605614712071 2020-11-17 12:19:49,057 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614791690 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605614791690 2020-11-17 12:19:49,065 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614884184 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605614884184 2020-11-17 12:19:49,071 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605614958342 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605614958342 2020-11-17 12:19:49,075 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615065692 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605615065692 2020-11-17 12:19:49,080 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615134617 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605615134617 2020-11-17 12:19:49,088 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615230492 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605615230492 2020-11-17 12:19:52,698 INFO [DataStreamer for file /hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/bfb26c7f7634492e8fd28f8bd6bbde1f] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:57,643 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:59,607 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#848 average throughput is 56.25 MB/second, slept 95 time(s) and total slept time is 21510 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 12:19:59,653 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:19:59,672 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of 73aae5715542e1a5370f7540d2117abc into bfb26c7f7634492e8fd28f8bd6bbde1f(size=679.0 M), total size for store is 12.7 G. This selection was in queue for 0sec, and took 1mins, 47sec to execute. 2020-11-17 12:19:59,672 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605615491980; duration=1mins, 47sec 2020-11-17 12:20:38,601 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:20:44,883 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:21:05,926 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615588989 with entries=23456, filesize=129.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615665894 2020-11-17 12:21:21,327 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.36 MB heapSize=128.58 MB 2020-11-17 12:21:21,376 INFO [Thread-3856] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:21:22,166 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.36 MB at sequenceid=264111048 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/145cba26da684a4bbb197e87f0908418 2020-11-17 12:21:22,173 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:21:22,196 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/145cba26da684a4bbb197e87f0908418, entries=363591, sequenceid=264111048, filesize=8.1 M 2020-11-17 12:21:22,198 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.36 MB/107336707, heapSize ~128.58 MB/134825184, currentSize=356.21 KB/364763 for e5dc5c620be19b02666a19c280ff582e in 871ms, sequenceid=264111048, compaction requested=true 2020-11-17 12:21:35,739 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.30 MB heapSize=128.06 MB 2020-11-17 12:21:35,774 INFO [Thread-3858] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:21:36,398 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.30 MB at sequenceid=252445338 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/f3937b0ae99048868d47d648355ac790 2020-11-17 12:21:36,405 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:21:36,423 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/f3937b0ae99048868d47d648355ac790, entries=311010, sequenceid=252445338, filesize=6.4 M 2020-11-17 12:21:36,425 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.30 MB/111463209, heapSize ~128.06 MB/134278096, currentSize=764.50 KB/782848 for 82421ce8c5c8b48c573baee83caa8305 in 686ms, sequenceid=252445338, compaction requested=true 2020-11-17 12:21:36,426 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 12:21:36,426 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/8340445874f44ee285f39a95e1d324ba, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/be212332b7e44537add2c5912b98d0bc, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/f3937b0ae99048868d47d648355ac790] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=45.5 M 2020-11-17 12:21:36,468 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:21:36,470 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:21:36,499 INFO [Thread-3860] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:21:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 56551 ns. 2020-11-17 12:21:48,459 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#852 average throughput is 61.26 MB/second, slept 13 time(s) and total slept time is 4958 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 12:21:48,537 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into f0b3985fdede4b10858d1056e9459f63(size=45.4 M), total size for store is 11.4 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 12:21:48,537 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605615696425; duration=12sec 2020-11-17 12:22:33,939 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615665894 with entries=27294, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615753908 2020-11-17 12:22:39,659 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 48177 ns. 2020-11-17 12:22:43,591 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:23:14,847 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:23:18,085 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:23:39,036 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 58358 ns. 2020-11-17 12:23:39,036 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 166043 ns. 2020-11-17 12:23:39,037 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 806562 ns. 2020-11-17 12:23:56,710 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615753908 with entries=24756, filesize=128.08 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615836676 2020-11-17 12:23:58,019 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:24:08,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 11108 ns. 2020-11-17 12:24:08,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 29548 ns. 2020-11-17 12:24:11,739 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:24:11,832 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:24:18,035 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:24:25,401 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:24:32,335 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:24:36,026 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=180.02 MB, max=1.83 GB, blockCount=43915, accesses=36866813, hits=31416637, hitRatio=85.22%, , cachingAccesses=31468626, cachingHits=31410126, cachingHitsRatio=99.81%, evictions=4649, evicted=0, evictedPerRun=0.0 2020-11-17 12:25:33,808 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:25:38,867 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615836676 with entries=31606, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615938842 2020-11-17 12:25:40,818 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:25:53,331 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.45 MB heapSize=128.19 MB 2020-11-17 12:25:53,364 INFO [Thread-3872] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:25:54,335 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.45 MB at sequenceid=264122524 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/b9e7729011b2411389436379a1af2836 2020-11-17 12:25:54,360 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b9e7729011b2411389436379a1af2836, entries=399510, sequenceid=264122524, filesize=8.0 M 2020-11-17 12:25:54,362 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.45 MB/104280708, heapSize ~128.19 MB/134421472, currentSize=585.78 KB/599837 for e5dc5c620be19b02666a19c280ff582e in 1031ms, sequenceid=264122524, compaction requested=true 2020-11-17 12:25:54,363 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 12:25:54,363 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/21f8ccc3d6a743d8b24ddf8876d6b9a9, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/145cba26da684a4bbb197e87f0908418, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b9e7729011b2411389436379a1af2836] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.2 M 2020-11-17 12:25:54,374 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:25:54,395 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:25:54,423 INFO [Thread-3874] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:25:54,676 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:25:59,073 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:26:00,114 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#854 average throughput is 53.25 MB/second, slept 3 time(s) and total slept time is 1026 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 12:26:00,178 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 9025a36e1d2e4995a5b144aae5d134ba(size=25.1 M), total size for store is 49.9 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 12:26:00,178 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605615954362; duration=5sec 2020-11-17 12:26:10,678 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:26:13,820 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.51 MB heapSize=128.21 MB 2020-11-17 12:26:13,851 INFO [Thread-3877] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:26:14,565 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.51 MB at sequenceid=252451446 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/1d581d6fcb0e43b3b2bfa21079d1df27 2020-11-17 12:26:14,573 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:26:14,591 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/1d581d6fcb0e43b3b2bfa21079d1df27, entries=310347, sequenceid=252451446, filesize=6.3 M 2020-11-17 12:26:14,593 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.51 MB/111686899, heapSize ~128.21 MB/134440256, currentSize=45.31 KB/46398 for 82421ce8c5c8b48c573baee83caa8305 in 774ms, sequenceid=252451446, compaction requested=true 2020-11-17 12:26:30,017 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:26:38,412 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:26:39,614 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 39607 ns. 2020-11-17 12:26:48,095 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615938842 with entries=23666, filesize=128.89 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616007789 2020-11-17 12:26:50,403 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.42 MB heapSize=128.04 MB 2020-11-17 12:26:50,439 INFO [Thread-3882] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:26:51,164 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.42 MB at sequenceid=260838393 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/8068f452b4cd46bd92602c34700a78be 2020-11-17 12:26:51,170 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:26:51,188 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/8068f452b4cd46bd92602c34700a78be, entries=371389, sequenceid=260838393, filesize=7.0 M 2020-11-17 12:26:51,189 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.42 MB/106343650, heapSize ~128.04 MB/134256072, currentSize=88.43 KB/90555 for 06f7ca9534710e37d90d9f8345864259 in 786ms, sequenceid=260838393, compaction requested=true 2020-11-17 12:26:51,189 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 12:26:51,190 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/1737f9ae6d994c2aad724c84852f2b89, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/bd5f05c2d48a4e8d8bbb2cef14ec3355, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/8068f452b4cd46bd92602c34700a78be] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=96.9 M 2020-11-17 12:26:51,216 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:26:51,384 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:26:51,403 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:26:51,430 INFO [Thread-3884] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:26:59,652 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:26:59,657 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:27:00,462 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:27:01,370 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:27:02,831 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:27:07,035 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:27:13,507 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:27:14,205 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#857 average throughput is 56.56 MB/second, slept 22 time(s) and total slept time is 6996 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 12:27:14,281 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into e212eaf57a8e424c8b34d7ba6a40ba2b(size=97.0 M), total size for store is 55.6 G. This selection was in queue for 0sec, and took 23sec to execute. 2020-11-17 12:27:14,281 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605616011189; duration=23sec 2020-11-17 12:27:21,212 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:27:31,628 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:27:34,059 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:27:54,588 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:28:03,539 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:28:12,742 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:28:32,670 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616007789 with entries=38633, filesize=128.66 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616112612 2020-11-17 12:28:39,633 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 58537 ns. 2020-11-17 12:28:44,935 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:28:48,413 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:29:06,815 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:29:08,827 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 32666 ns. 2020-11-17 12:29:08,827 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 19659 ns. 2020-11-17 12:29:21,528 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.27 MB heapSize=128.06 MB 2020-11-17 12:29:21,571 INFO [Thread-3893] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:29:22,349 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.27 MB at sequenceid=264132872 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/3cafb2c9d9684739b6f354b432334694 2020-11-17 12:29:22,355 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:29:22,373 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3cafb2c9d9684739b6f354b432334694, entries=371583, sequenceid=264132872, filesize=8.1 M 2020-11-17 12:29:22,374 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.27 MB/106193780, heapSize ~128.06 MB/134284672, currentSize=714.73 KB/731882 for e5dc5c620be19b02666a19c280ff582e in 846ms, sequenceid=264132872, compaction requested=true 2020-11-17 12:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.48 MB, max=1.83 GB, blockCount=44128, accesses=37686353, hits=32209986, hitRatio=85.47%, , cachingAccesses=32262224, cachingHits=32203458, cachingHitsRatio=99.82%, evictions=4679, evicted=0, evictedPerRun=0.0 2020-11-17 12:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 12:29:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 49330 ns. 2020-11-17 12:29:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 169354 ns. 2020-11-17 12:29:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 471162 ns. 2020-11-17 12:29:38,844 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616112612 with entries=24586, filesize=128.22 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616178822 2020-11-17 12:30:51,832 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.48 MB heapSize=128.27 MB 2020-11-17 12:30:51,872 INFO [Thread-3898] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:30:52,514 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.48 MB at sequenceid=252459613 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/6f5481bf9c554ad194a10113b14a33c1 2020-11-17 12:30:52,523 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:30:52,550 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/6f5481bf9c554ad194a10113b14a33c1, entries=311487, sequenceid=252459613, filesize=6.3 M 2020-11-17 12:30:52,551 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.48 MB/111654543, heapSize ~128.27 MB/134500400, currentSize=229.75 KB/235262 for 82421ce8c5c8b48c573baee83caa8305 in 719ms, sequenceid=252459613, compaction requested=true 2020-11-17 12:30:52,552 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 12:30:52,552 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/f0b3985fdede4b10858d1056e9459f63, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/1d581d6fcb0e43b3b2bfa21079d1df27, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/6f5481bf9c554ad194a10113b14a33c1] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=58.1 M 2020-11-17 12:30:52,582 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:30:52,584 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:30:52,610 INFO [Thread-3900] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:31:07,752 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#860 average throughput is 62.46 MB/second, slept 18 time(s) and total slept time is 6574 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 12:31:07,839 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 445dc876927b457bbc5931e291cee1d5(size=58.0 M), total size for store is 11.5 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 12:31:07,839 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605616252551; duration=15sec 2020-11-17 12:31:15,987 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:31:19,953 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.75 MB heapSize=128.00 MB 2020-11-17 12:31:19,989 INFO [Thread-3903] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:31:20,581 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616178822 with entries=38661, filesize=128.28 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616280558 2020-11-17 12:31:20,783 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.75 MB at sequenceid=272925463 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/c46838f24e144aaa982c2c39ae0e51f9 2020-11-17 12:31:20,790 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:31:20,809 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/c46838f24e144aaa982c2c39ae0e51f9, entries=319797, sequenceid=272925463, filesize=8.7 M 2020-11-17 12:31:20,810 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.75 MB/109834095, heapSize ~128.00 MB/134222256, currentSize=17.53 KB/17954 for 0469ce862509756add4a8cdb00bc465c in 857ms, sequenceid=272925463, compaction requested=true 2020-11-17 12:31:20,811 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 12:31:20,811 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/23a73dea055c483f87faaafe60b6b92b, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/9e5d8bf8d2b64e74be502b52855f0f8f, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/95e0cc948570482cacba08ca7eb15343, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/c46838f24e144aaa982c2c39ae0e51f9] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=403.1 M 2020-11-17 12:31:20,871 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:31:20,874 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:31:20,878 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:31:20,897 INFO [Thread-3905] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:31:25,272 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:31:43,141 INFO [DataStreamer for file /hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/8176efafa5ef441780f9882e7231c0bf] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:31:49,741 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:32:10,353 INFO [DataStreamer for file /hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/8176efafa5ef441780f9882e7231c0bf] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:32:23,160 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:32:23,921 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:32:25,116 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616280558 with entries=24174, filesize=128.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616345070 2020-11-17 12:32:28,851 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:32:28,854 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:32:29,096 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:32:29,208 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:32:29,210 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:32:29,213 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:32:29,217 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:32:29,222 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:32:31,533 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:32:33,793 INFO [DataStreamer for file /hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/8176efafa5ef441780f9882e7231c0bf] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:32:37,050 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#862 average throughput is 60.60 MB/second, slept 83 time(s) and total slept time is 34730 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 12:32:37,131 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 8176efafa5ef441780f9882e7231c0bf(size=399.6 M), total size for store is 43.0 G. This selection was in queue for 0sec, and took 1mins, 16sec to execute. 2020-11-17 12:32:37,131 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605616280811; duration=1mins, 16sec 2020-11-17 12:32:38,791 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 39655 ns. 2020-11-17 12:32:48,537 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:33:18,076 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:33:44,397 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616345070 with entries=29263, filesize=128.32 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616424371 2020-11-17 12:33:46,131 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.71 MB heapSize=128.01 MB 2020-11-17 12:33:46,166 INFO [Thread-3916] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:33:46,922 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.71 MB at sequenceid=264146386 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/bd2be50275c542a49939d2df1fb5792f 2020-11-17 12:33:46,928 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:33:46,943 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/bd2be50275c542a49939d2df1fb5792f, entries=364368, sequenceid=264146386, filesize=8.0 M 2020-11-17 12:33:46,945 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.71 MB/106653022, heapSize ~128.01 MB/134232136, currentSize=56.39 KB/57747 for e5dc5c620be19b02666a19c280ff582e in 814ms, sequenceid=264146386, compaction requested=true 2020-11-17 12:33:46,946 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 12:33:46,946 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/9025a36e1d2e4995a5b144aae5d134ba, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3cafb2c9d9684739b6f354b432334694, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/bd2be50275c542a49939d2df1fb5792f] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.2 M 2020-11-17 12:33:46,974 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:33:46,976 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:33:47,001 INFO [Thread-3918] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:33:55,219 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#864 average throughput is 61.55 MB/second, slept 10 time(s) and total slept time is 2464 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 12:33:55,274 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 1afde67da0a04e28ac77307b7b1455be(size=42.4 M), total size for store is 50.0 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 12:33:55,274 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605616426945; duration=8sec 2020-11-17 12:34:01,254 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.73 MB heapSize=128.26 MB 2020-11-17 12:34:01,281 INFO [Thread-3920] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:34:01,841 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.73 MB at sequenceid=252465348 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/1df77ea0bdb64c408471ec2a91bee721 2020-11-17 12:34:01,847 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:34:01,865 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/1df77ea0bdb64c408471ec2a91bee721, entries=307847, sequenceid=252465348, filesize=6.2 M 2020-11-17 12:34:01,866 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.73 MB/111919600, heapSize ~128.26 MB/134485400, currentSize=355.46 KB/363994 for 82421ce8c5c8b48c573baee83caa8305 in 612ms, sequenceid=252465348, compaction requested=true 2020-11-17 12:34:18,993 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15015 ns. 2020-11-17 12:34:18,993 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 28952 ns. 2020-11-17 12:34:33,282 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.94 MB heapSize=128.37 MB 2020-11-17 12:34:33,318 INFO [Thread-3923] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:34:34,024 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.94 MB at sequenceid=273461895 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/059d21b889cb4812b594b8169c752f7f 2020-11-17 12:34:34,030 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:34:34,047 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/059d21b889cb4812b594b8169c752f7f, entries=280600, sequenceid=273461895, filesize=8.4 M 2020-11-17 12:34:34,049 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~107.94 MB/113181997, heapSize ~128.37 MB/134607208, currentSize=351.14 KB/359572 for fc93f6dcc564341dcc983bf30971f73d in 767ms, sequenceid=273461895, compaction requested=true 2020-11-17 12:34:34,049 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 12:34:34,050 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/c3c28bdb02fe4e8c835f6e08a468f7e7, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/6864873a0593460489ad34c51b883dcf, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/059d21b889cb4812b594b8169c752f7f] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=43.0 M 2020-11-17 12:34:34,080 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:34:34,084 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:34:34,109 INFO [Thread-3925] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=175.91 MB, max=1.83 GB, blockCount=44079, accesses=37788691, hits=32211788, hitRatio=85.24%, , cachingAccesses=32263993, cachingHits=32205201, cachingHitsRatio=99.82%, evictions=4709, evicted=0, evictedPerRun=0.0 2020-11-17 12:34:38,764 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 56500 ns. 2020-11-17 12:34:39,045 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.54 MB heapSize=128.00 MB 2020-11-17 12:34:39,081 INFO [Thread-3927] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:34:40,342 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.54 MB at sequenceid=262707037 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/bc2d2247c7234ec6b513d7e6278ced71 2020-11-17 12:34:40,347 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:34:40,363 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/bc2d2247c7234ec6b513d7e6278ced71, entries=585160, sequenceid=262707037, filesize=8.9 M 2020-11-17 12:34:40,364 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.54 MB/89690686, heapSize ~128 MB/134217728, currentSize=131.78 KB/134939 for 73aae5715542e1a5370f7540d2117abc in 1319ms, sequenceid=262707037, compaction requested=true 2020-11-17 12:34:42,725 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#867 average throughput is 62.50 MB/second, slept 10 time(s) and total slept time is 4005 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 12:34:42,779 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into e6d5fc863e1c457e86c899d41372736d(size=43.5 M), total size for store is 33.6 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 12:34:42,779 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605616474049; duration=8sec 2020-11-17 12:34:57,932 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.98 MB heapSize=128.00 MB 2020-11-17 12:34:57,971 INFO [Thread-3930] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:34:58,767 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616424371 with entries=27640, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616498749 2020-11-17 12:34:58,786 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.98 MB at sequenceid=260863821 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/71f322ee909c426aa549de3637ea6bcf 2020-11-17 12:34:58,796 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:34:58,813 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/71f322ee909c426aa549de3637ea6bcf, entries=363134, sequenceid=260863821, filesize=6.8 M 2020-11-17 12:34:58,882 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.98 MB/106938610, heapSize ~128.00 MB/134219712, currentSize=204.26 KB/209166 for 06f7ca9534710e37d90d9f8345864259 in 949ms, sequenceid=260863821, compaction requested=true 2020-11-17 12:35:32,170 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:35:39,580 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 367354 ns. 2020-11-17 12:35:39,581 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 156689 ns. 2020-11-17 12:35:39,581 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 47495 ns. 2020-11-17 12:36:06,580 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616498749 with entries=25188, filesize=128.19 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616566542 2020-11-17 12:36:48,016 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:37:11,551 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.78 MB heapSize=128.43 MB 2020-11-17 12:37:11,595 INFO [Thread-3937] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:37:12,132 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.78 MB at sequenceid=252471084 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/0ab3deb2116c4211bbdc226a457ee801 2020-11-17 12:37:12,139 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:37:12,163 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0ab3deb2116c4211bbdc226a457ee801, entries=309658, sequenceid=252471084, filesize=6.2 M 2020-11-17 12:37:12,165 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.78 MB/111961965, heapSize ~128.43 MB/134670376, currentSize=248.47 KB/254430 for 82421ce8c5c8b48c573baee83caa8305 in 614ms, sequenceid=252471084, compaction requested=true 2020-11-17 12:37:12,166 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 12:37:12,166 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/445dc876927b457bbc5931e291cee1d5, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/1df77ea0bdb64c408471ec2a91bee721, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0ab3deb2116c4211bbdc226a457ee801] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=70.4 M 2020-11-17 12:37:12,210 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:37:12,213 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:37:12,242 INFO [Thread-3939] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:37:30,002 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616566542 with entries=31439, filesize=128.22 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616649957 2020-11-17 12:37:32,312 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#871 average throughput is 57.76 MB/second, slept 23 time(s) and total slept time is 9832 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 12:37:32,377 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:37:32,395 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 3f494fc3428441fea390a593e7f0caf1(size=70.4 M), total size for store is 11.5 G. This selection was in queue for 0sec, and took 20sec to execute. 2020-11-17 12:37:32,395 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605616632165; duration=20sec 2020-11-17 12:37:39,554 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 90591 ns. 2020-11-17 12:37:47,305 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=97.35 MB heapSize=128.08 MB 2020-11-17 12:37:47,333 INFO [Thread-3942] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:37:48,181 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.35 MB at sequenceid=264158681 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/3ec52843df01451c85f6d4af15dd48bc 2020-11-17 12:37:48,188 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:37:48,258 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3ec52843df01451c85f6d4af15dd48bc, entries=427353, sequenceid=264158681, filesize=7.9 M 2020-11-17 12:37:48,266 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~97.35 MB/102074017, heapSize ~128.08 MB/134300408, currentSize=416.66 KB/426663 for e5dc5c620be19b02666a19c280ff582e in 961ms, sequenceid=264158681, compaction requested=true 2020-11-17 12:38:35,813 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.16 MB heapSize=128.00 MB 2020-11-17 12:38:35,844 INFO [Thread-3946] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:38:36,916 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.16 MB at sequenceid=288589084 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/9f44df3f75a54e3bb6c50391efd853bf 2020-11-17 12:38:36,921 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:38:36,940 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/9f44df3f75a54e3bb6c50391efd853bf, entries=529041, sequenceid=288589084, filesize=8.9 M 2020-11-17 12:38:36,941 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.16 MB/94538580, heapSize ~128 MB/134217728, currentSize=41.25 KB/42238 for 5e5f05205d7e40b0141cd977dd009ba1 in 1128ms, sequenceid=288589084, compaction requested=true 2020-11-17 12:38:43,141 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616649957 with entries=27398, filesize=128.28 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616723122 2020-11-17 12:38:43,141 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615309648 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605615309648 2020-11-17 12:39:19,227 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 22355 ns. 2020-11-17 12:39:19,228 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 24882 ns. 2020-11-17 12:39:25,445 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:39:31,350 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=176.33 MB, max=1.83 GB, blockCount=44072, accesses=37820222, hits=32218053, hitRatio=85.19%, , cachingAccesses=32270278, cachingHits=32211463, cachingHitsRatio=99.82%, evictions=4739, evicted=0, evictedPerRun=0.0 2020-11-17 12:39:39,687 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:40:38,754 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 73873 ns. 2020-11-17 12:40:57,230 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616723122 with entries=49620, filesize=128.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616857185 2020-11-17 12:41:21,749 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.59 MB heapSize=128.09 MB 2020-11-17 12:41:21,801 INFO [Thread-3954] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:41:22,588 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.59 MB at sequenceid=264169618 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/8c0afa08ee484a8f84e446b8d4f38ef1 2020-11-17 12:41:22,601 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:41:22,627 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8c0afa08ee484a8f84e446b8d4f38ef1, entries=353605, sequenceid=264169618, filesize=8.3 M 2020-11-17 12:41:22,629 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.59 MB/107575412, heapSize ~128.09 MB/134314032, currentSize=1.84 MB/1927298 for e5dc5c620be19b02666a19c280ff582e in 880ms, sequenceid=264169618, compaction requested=true 2020-11-17 12:41:22,631 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 12:41:22,631 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1afde67da0a04e28ac77307b7b1455be, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3ec52843df01451c85f6d4af15dd48bc, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8c0afa08ee484a8f84e446b8d4f38ef1] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=58.5 M 2020-11-17 12:41:22,671 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:41:22,674 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:41:22,706 INFO [Thread-3956] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:41:34,431 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#875 average throughput is 60.18 MB/second, slept 13 time(s) and total slept time is 3810 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 12:41:34,502 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into b6e064673e2146fb9bea2d1fe8e32320(size=60.0 M), total size for store is 50.0 G. This selection was in queue for 0sec, and took 11sec to execute. 2020-11-17 12:41:34,502 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605616882629; duration=11sec 2020-11-17 12:41:38,832 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 63482 ns. 2020-11-17 12:41:38,832 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 159887 ns. 2020-11-17 12:41:38,833 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 404435 ns. 2020-11-17 12:41:43,360 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:42:51,245 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616857185 with entries=41982, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616971195 2020-11-17 12:43:02,082 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:43:38,750 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 53532 ns. 2020-11-17 12:44:28,725 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 10766 ns. 2020-11-17 12:44:28,725 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 26746 ns. 2020-11-17 12:44:32,898 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616971195 with entries=38689, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617072849 2020-11-17 12:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=176.14 MB, max=1.83 GB, blockCount=44075, accesses=38137159, hits=32523496, hitRatio=85.28%, , cachingAccesses=32575724, cachingHits=32516900, cachingHitsRatio=99.82%, evictions=4769, evicted=0, evictedPerRun=0.0 2020-11-17 12:44:44,201 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.38 MB heapSize=128.66 MB 2020-11-17 12:44:44,239 INFO [Thread-3968] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:44:44,971 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.38 MB at sequenceid=272964415 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/ea598106cac548ec96d3c9d272a5eda1 2020-11-17 12:44:44,978 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:44:44,994 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/ea598106cac548ec96d3c9d272a5eda1, entries=320879, sequenceid=272964415, filesize=8.7 M 2020-11-17 12:44:44,995 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.38 MB/110494625, heapSize ~128.66 MB/134907520, currentSize=15.42 KB/15787 for 0469ce862509756add4a8cdb00bc465c in 794ms, sequenceid=272964415, compaction requested=true 2020-11-17 12:44:58,152 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:45:11,182 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.64 MB heapSize=128.01 MB 2020-11-17 12:45:11,217 INFO [Thread-3971] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:45:12,105 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.64 MB at sequenceid=264181094 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/9de8a363bd3e4a949dc85a1430d2a2b2 2020-11-17 12:45:12,114 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:45:12,140 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/9de8a363bd3e4a949dc85a1430d2a2b2, entries=351962, sequenceid=264181094, filesize=8.1 M 2020-11-17 12:45:12,142 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.64 MB/107629481, heapSize ~128.01 MB/134224864, currentSize=111.58 KB/114262 for e5dc5c620be19b02666a19c280ff582e in 960ms, sequenceid=264181094, compaction requested=true 2020-11-17 12:45:16,011 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:45:24,532 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:45:29,744 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:45:46,835 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=87.76 MB heapSize=128.00 MB 2020-11-17 12:45:46,866 INFO [Thread-3975] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:45:47,881 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=87.76 MB at sequenceid=292440723 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/5f56686d2a3a4fd4a8833bad3eee8873 2020-11-17 12:45:47,888 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:45:47,905 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/5f56686d2a3a4fd4a8833bad3eee8873, entries=558360, sequenceid=292440723, filesize=8.6 M 2020-11-17 12:45:47,907 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~87.76 MB/92027817, heapSize ~128.00 MB/134218040, currentSize=50.99 KB/52209 for bdaa31a63b7f3d06d9909b7560333e79 in 1071ms, sequenceid=292440723, compaction requested=true 2020-11-17 12:45:58,487 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:46:21,311 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:46:28,684 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.90 MB heapSize=128.09 MB 2020-11-17 12:46:28,720 INFO [Thread-3979] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:46:29,478 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.90 MB at sequenceid=252487696 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/a26f0a68601e4aa687245c08bc9657b7 2020-11-17 12:46:29,484 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:46:29,499 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a26f0a68601e4aa687245c08bc9657b7, entries=316920, sequenceid=252487696, filesize=6.7 M 2020-11-17 12:46:29,500 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.90 MB/111041548, heapSize ~128.09 MB/134315712, currentSize=190.36 KB/194925 for 82421ce8c5c8b48c573baee83caa8305 in 816ms, sequenceid=252487696, compaction requested=true 2020-11-17 12:46:35,242 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617072849 with entries=45909, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617195219 2020-11-17 12:46:35,243 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615405568 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605615405568 2020-11-17 12:46:35,254 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615481188 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605615481188 2020-11-17 12:46:35,261 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615588989 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605615588989 2020-11-17 12:46:35,266 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615665894 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605615665894 2020-11-17 12:46:35,271 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615753908 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605615753908 2020-11-17 12:46:35,276 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615836676 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605615836676 2020-11-17 12:46:35,281 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605615938842 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605615938842 2020-11-17 12:46:35,286 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616007789 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605616007789 2020-11-17 12:46:35,291 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616112612 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605616112612 2020-11-17 12:46:35,296 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616178822 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605616178822 2020-11-17 12:46:35,301 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616280558 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605616280558 2020-11-17 12:46:35,306 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616345070 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605616345070 2020-11-17 12:46:38,671 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 50029 ns. 2020-11-17 12:46:44,373 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:47:06,467 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:47:09,907 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:47:26,706 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:47:38,960 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 179237 ns. 2020-11-17 12:47:38,961 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 362090 ns. 2020-11-17 12:47:38,961 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 68172 ns. 2020-11-17 12:47:57,603 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.19 MB heapSize=128.00 MB 2020-11-17 12:47:57,650 INFO [Thread-3985] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:47:58,487 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.19 MB at sequenceid=260904719 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/a018fc602e704247b3fd64453fcd9a3a 2020-11-17 12:47:58,493 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:47:58,509 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/a018fc602e704247b3fd64453fcd9a3a, entries=382655, sequenceid=260904719, filesize=7.3 M 2020-11-17 12:47:58,510 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.19 MB/105059941, heapSize ~128.00 MB/134217832, currentSize=84.37 KB/86399 for 06f7ca9534710e37d90d9f8345864259 in 907ms, sequenceid=260904719, compaction requested=true 2020-11-17 12:47:58,512 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 12:47:58,512 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/c1b76172f58f437db90b4219490dbfeb, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/e212eaf57a8e424c8b34d7ba6a40ba2b, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/71f322ee909c426aa549de3637ea6bcf, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/a018fc602e704247b3fd64453fcd9a3a] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=236.9 M 2020-11-17 12:47:58,552 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:47:58,557 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:47:58,559 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:47:58,593 INFO [Thread-3987] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:00,695 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:03,001 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:06,545 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:06,547 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:14,505 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:14,879 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:14,881 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:15,002 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:15,443 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:17,266 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617195219 with entries=38589, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617297235 2020-11-17 12:48:18,093 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:18,097 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:19,045 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:19,047 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:19,048 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:19,050 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:22,076 INFO [DataStreamer for file /hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/4a79d8c14d0b4961ad1b865fd166a68f] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:48:39,040 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 36887 ns. 2020-11-17 12:48:49,820 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#881 average throughput is 60.70 MB/second, slept 57 time(s) and total slept time is 19572 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 12:48:49,958 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 4a79d8c14d0b4961ad1b865fd166a68f(size=232.7 M), total size for store is 55.6 G. This selection was in queue for 0sec, and took 51sec to execute. 2020-11-17 12:48:49,958 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605617278511; duration=51sec 2020-11-17 12:49:29,530 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 35156 ns. 2020-11-17 12:49:29,530 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 11724 ns. 2020-11-17 12:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=174.48 MB, max=1.83 GB, blockCount=44106, accesses=38193499, hits=32529193, hitRatio=85.17%, , cachingAccesses=32581423, cachingHits=32522568, cachingHitsRatio=99.82%, evictions=4799, evicted=0, evictedPerRun=0.0 2020-11-17 12:50:03,784 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.04 MB heapSize=128.29 MB 2020-11-17 12:50:03,808 INFO [Thread-3996] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:50:04,523 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.04 MB at sequenceid=273491035 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/9af61bd08369489cbdcab4d7cf7498ea 2020-11-17 12:50:04,528 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:50:04,530 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=97.99 MB heapSize=128.02 MB 2020-11-17 12:50:04,546 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/9af61bd08369489cbdcab4d7cf7498ea, entries=277785, sequenceid=273491035, filesize=8.5 M 2020-11-17 12:50:04,549 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.04 MB/113289824, heapSize ~128.29 MB/134519232, currentSize=14.31 KB/14655 for fc93f6dcc564341dcc983bf30971f73d in 765ms, sequenceid=273491035, compaction requested=true 2020-11-17 12:50:04,555 INFO [Thread-3998] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:50:05,413 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.99 MB at sequenceid=264196236 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/05a67ce3f8f0401b9139c99b13d2e9d4 2020-11-17 12:50:05,434 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/05a67ce3f8f0401b9139c99b13d2e9d4, entries=418459, sequenceid=264196236, filesize=8.0 M 2020-11-17 12:50:05,436 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~97.99 MB/102751634, heapSize ~128.02 MB/134239904, currentSize=425.61 KB/435823 for e5dc5c620be19b02666a19c280ff582e in 906ms, sequenceid=264196236, compaction requested=true 2020-11-17 12:50:05,437 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 12:50:05,437 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/b6e064673e2146fb9bea2d1fe8e32320, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/9de8a363bd3e4a949dc85a1430d2a2b2, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/05a67ce3f8f0401b9139c99b13d2e9d4] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=76.1 M 2020-11-17 12:50:05,473 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:50:05,475 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:50:05,505 INFO [Thread-4000] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:50:19,245 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617297235 with entries=45151, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617419201 2020-11-17 12:50:20,315 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#884 average throughput is 61.19 MB/second, slept 16 time(s) and total slept time is 5264 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 12:50:20,378 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into c94e336c25bd4e2a96c23cc4817a7570(size=77.8 M), total size for store is 50.0 G. This selection was in queue for 0sec, and took 14sec to execute. 2020-11-17 12:50:20,378 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605617405437; duration=14sec 2020-11-17 12:50:40,005 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:50:47,767 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.63 MB heapSize=128.11 MB 2020-11-17 12:50:47,807 INFO [Thread-4005] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:50:48,841 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.63 MB at sequenceid=262757442 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/ad01bb4b93f741eb925cd488c67f00ae 2020-11-17 12:50:48,847 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:50:48,873 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/ad01bb4b93f741eb925cd488c67f00ae, entries=580120, sequenceid=262757442, filesize=9.0 M 2020-11-17 12:50:48,874 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.63 MB/89792773, heapSize ~128.11 MB/134336560, currentSize=246.11 KB/252019 for 73aae5715542e1a5370f7540d2117abc in 1107ms, sequenceid=262757442, compaction requested=true 2020-11-17 12:51:04,722 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:51:38,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 56888 ns. 2020-11-17 12:52:12,495 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617419201 with entries=42068, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617532446 2020-11-17 12:52:12,495 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616424371 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605616424371 2020-11-17 12:52:12,543 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616498749 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605616498749 2020-11-17 12:52:12,562 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616566542 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605616566542 2020-11-17 12:53:18,095 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:53:33,544 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.67 MB heapSize=128.32 MB 2020-11-17 12:53:33,577 INFO [Thread-4014] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:53:34,308 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.67 MB at sequenceid=264206937 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/f9fbf553496b4bf8a43cbbe8c5576a77 2020-11-17 12:53:34,315 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:53:34,332 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f9fbf553496b4bf8a43cbbe8c5576a77, entries=355778, sequenceid=264206937, filesize=8.2 M 2020-11-17 12:53:34,334 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.67 MB/107656745, heapSize ~128.32 MB/134548208, currentSize=599.53 KB/613921 for e5dc5c620be19b02666a19c280ff582e in 790ms, sequenceid=264206937, compaction requested=true 2020-11-17 12:53:38,709 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 189845 ns. 2020-11-17 12:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 390613 ns. 2020-11-17 12:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 52797 ns. 2020-11-17 12:53:43,266 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617532446 with entries=33718, filesize=128.00 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617623242 2020-11-17 12:54:29,575 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 33643 ns. 2020-11-17 12:54:29,575 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15553 ns. 2020-11-17 12:54:36,025 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=176.41 MB, max=1.83 GB, blockCount=44077, accesses=38210545, hits=32531461, hitRatio=85.14%, , cachingAccesses=32583721, cachingHits=32524833, cachingHitsRatio=99.82%, evictions=4829, evicted=0, evictedPerRun=0.0 2020-11-17 12:54:39,074 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 32875 ns. 2020-11-17 12:54:48,343 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:54:55,107 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:54:58,205 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:55:04,648 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:55:54,411 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617623242 with entries=48325, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617754359 2020-11-17 12:56:18,227 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:56:38,772 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 48812 ns. 2020-11-17 12:57:02,602 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:57:21,240 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.12 MB heapSize=128.00 MB 2020-11-17 12:57:21,289 INFO [Thread-4025] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:57:22,199 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.12 MB at sequenceid=264218439 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a7e06d275da14a3b95054665def29b9c 2020-11-17 12:57:22,206 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:57:22,223 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a7e06d275da14a3b95054665def29b9c, entries=345010, sequenceid=264218439, filesize=8.1 M 2020-11-17 12:57:22,227 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.12 MB/108124949, heapSize ~128.00 MB/134218840, currentSize=49.53 KB/50716 for e5dc5c620be19b02666a19c280ff582e in 987ms, sequenceid=264218439, compaction requested=true 2020-11-17 12:57:22,228 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 12:57:22,229 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c94e336c25bd4e2a96c23cc4817a7570, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f9fbf553496b4bf8a43cbbe8c5576a77, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/a7e06d275da14a3b95054665def29b9c] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=94.2 M 2020-11-17 12:57:22,277 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:57:22,280 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:57:22,305 INFO [Thread-4027] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:57:33,767 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617754359 with entries=37107, filesize=128.20 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617853701 2020-11-17 12:57:40,472 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#888 average throughput is 61.23 MB/second, slept 21 time(s) and total slept time is 5088 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 12:57:40,541 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into ec7e959c51a54c389c4a75a346e7ddfe(size=95.5 M), total size for store is 50.0 G. This selection was in queue for 0sec, and took 18sec to execute. 2020-11-17 12:57:40,541 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605617842227; duration=18sec 2020-11-17 12:57:48,090 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.57 MB heapSize=128.06 MB 2020-11-17 12:57:48,116 INFO [Thread-4030] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:57:49,062 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.57 MB at sequenceid=252507835 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/e57bbbd35c65451da0e76cd1d90f2611 2020-11-17 12:57:49,067 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:57:49,086 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/e57bbbd35c65451da0e76cd1d90f2611, entries=321010, sequenceid=252507835, filesize=6.8 M 2020-11-17 12:57:49,088 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.57 MB/110699250, heapSize ~128.06 MB/134281712, currentSize=114.33 KB/117070 for 82421ce8c5c8b48c573baee83caa8305 in 998ms, sequenceid=252507835, compaction requested=true 2020-11-17 12:57:49,088 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 12:57:49,089 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/3f494fc3428441fea390a593e7f0caf1, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a26f0a68601e4aa687245c08bc9657b7, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/e57bbbd35c65451da0e76cd1d90f2611] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=83.9 M 2020-11-17 12:57:49,153 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:57:49,155 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:57:49,183 INFO [Thread-4032] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:57:58,855 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:58:08,126 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:58:08,137 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:58:08,690 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:58:08,692 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:58:10,543 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#890 average throughput is 64.14 MB/second, slept 25 time(s) and total slept time is 7166 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 12:58:10,623 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into daf25a986df34055806ddbafd51b5cdc(size=83.8 M), total size for store is 11.5 G. This selection was in queue for 0sec, and took 21sec to execute. 2020-11-17 12:58:10,623 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605617869088; duration=21sec 2020-11-17 12:58:34,194 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.39 MB heapSize=128.76 MB 2020-11-17 12:58:34,238 INFO [Thread-4036] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:58:34,777 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:58:35,104 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.39 MB at sequenceid=273005046 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/b9ffa0582dda40059fbe057144de5cbc 2020-11-17 12:58:35,126 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/b9ffa0582dda40059fbe057144de5cbc, entries=321284, sequenceid=273005046, filesize=8.7 M 2020-11-17 12:58:35,127 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.39 MB/110514628, heapSize ~128.76 MB/135017832, currentSize=33.32 KB/34115 for 0469ce862509756add4a8cdb00bc465c in 933ms, sequenceid=273005046, compaction requested=true 2020-11-17 12:58:38,398 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:59:12,006 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:59:28,085 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 12:59:32,761 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617853701 with entries=43830, filesize=128.29 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617972719 2020-11-17 12:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.66 GB, freeSize=173.82 MB, max=1.83 GB, blockCount=44132, accesses=38661763, hits=32942313, hitRatio=85.21%, , cachingAccesses=32994620, cachingHits=32935677, cachingHitsRatio=99.82%, evictions=4859, evicted=0, evictedPerRun=0.0 2020-11-17 12:59:39,060 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 423338 ns. 2020-11-17 12:59:39,060 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 187841 ns. 2020-11-17 12:59:39,060 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 11029 ns. 2020-11-17 12:59:39,060 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 26806 ns. 2020-11-17 12:59:39,060 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 54906 ns. 2020-11-17 12:59:53,050 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:00:21,036 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:00:31,530 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:00:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 39785 ns. 2020-11-17 13:00:45,242 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:00:48,791 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:00:55,631 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:01:00,459 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:01:07,191 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:01:15,417 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:01:24,634 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:01:31,968 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617972719 with entries=43456, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618091929 2020-11-17 13:01:49,489 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:01:54,324 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.34 MB heapSize=128.07 MB 2020-11-17 13:01:54,375 INFO [Thread-4046] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:01:55,281 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.34 MB at sequenceid=264232054 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/6962ad15beae48f18b2278c8874d4ff8 2020-11-17 13:01:55,288 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:01:55,307 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6962ad15beae48f18b2278c8874d4ff8, entries=414312, sequenceid=264232054, filesize=8.1 M 2020-11-17 13:01:55,309 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.34 MB/103116696, heapSize ~128.07 MB/134292560, currentSize=469.23 KB/480494 for e5dc5c620be19b02666a19c280ff582e in 985ms, sequenceid=264232054, compaction requested=true 2020-11-17 13:02:00,763 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:02:18,593 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:02:26,965 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:02:26,966 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:02:38,787 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 50411 ns. 2020-11-17 13:03:01,368 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618091929 with entries=32995, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618181334 2020-11-17 13:03:22,340 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:03:22,881 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.14 MB heapSize=128.01 MB 2020-11-17 13:03:22,925 INFO [Thread-4051] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:03:24,006 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.14 MB at sequenceid=288670448 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/5b0faf0ca399453d85de7dce8c26c2cb 2020-11-17 13:03:24,034 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/5b0faf0ca399453d85de7dce8c26c2cb, entries=528962, sequenceid=288670448, filesize=9.2 M 2020-11-17 13:03:24,035 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~90.14 MB/94515573, heapSize ~128.01 MB/134227120, currentSize=419.66 KB/429730 for 5e5f05205d7e40b0141cd977dd009ba1 in 1154ms, sequenceid=288670448, compaction requested=true 2020-11-17 13:03:24,036 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 13:03:24,037 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/1301cf27422041ab95866faa3d14f2c9, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/9f44df3f75a54e3bb6c50391efd853bf, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/5b0faf0ca399453d85de7dce8c26c2cb] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=27.1 M 2020-11-17 13:03:24,069 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:03:24,073 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:03:24,097 INFO [Thread-4053] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:03:28,578 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#894 average throughput is 60.10 MB/second, slept 4 time(s) and total slept time is 774 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 13:03:28,641 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into 745d0cf0e6a743dcbbfadc5a8100f779(size=27.5 M), total size for store is 24.4 G. This selection was in queue for 0sec, and took 4sec to execute. 2020-11-17 13:03:28,641 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=10, startTime=1605618204035; duration=4sec 2020-11-17 13:03:41,571 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:04:10,677 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.57 MB, max=1.83 GB, blockCount=44167, accesses=39245402, hits=33521463, hitRatio=85.42%, , cachingAccesses=33573821, cachingHits=33514813, cachingHitsRatio=99.82%, evictions=4889, evicted=0, evictedPerRun=0.0 2020-11-17 13:04:48,800 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 30007 ns. 2020-11-17 13:04:48,800 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 12312 ns. 2020-11-17 13:05:11,969 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:05:18,343 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618181334 with entries=51434, filesize=128.57 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618318304 2020-11-17 13:05:18,343 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616649957 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605616649957 2020-11-17 13:05:18,350 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616723122 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605616723122 2020-11-17 13:05:18,365 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616857185 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605616857185 2020-11-17 13:05:18,371 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605616971195 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605616971195 2020-11-17 13:05:23,372 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.94 MB heapSize=128.02 MB 2020-11-17 13:05:23,395 INFO [Thread-4059] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:05:24,177 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.94 MB at sequenceid=273519337 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/0be8789c6fae4da896ebd1e90a46532d 2020-11-17 13:05:24,182 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:05:24,202 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/0be8789c6fae4da896ebd1e90a46532d, entries=275368, sequenceid=273519337, filesize=8.5 M 2020-11-17 13:05:24,203 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~107.94 MB/113188061, heapSize ~128.02 MB/134239648, currentSize=18.65 KB/19093 for fc93f6dcc564341dcc983bf30971f73d in 831ms, sequenceid=273519337, compaction requested=true 2020-11-17 13:05:24,204 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 13:05:24,205 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/e6d5fc863e1c457e86c899d41372736d, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/9af61bd08369489cbdcab4d7cf7498ea, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/0be8789c6fae4da896ebd1e90a46532d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=60.5 M 2020-11-17 13:05:24,238 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:05:24,243 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:05:24,262 INFO [Thread-4061] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:05:26,815 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.42 MB heapSize=128.18 MB 2020-11-17 13:05:26,852 INFO [Thread-4063] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:05:27,624 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.42 MB at sequenceid=264242958 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/2480e1d2210a4362b11c3f7c1de5749d 2020-11-17 13:05:27,630 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:05:27,648 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2480e1d2210a4362b11c3f7c1de5749d, entries=371395, sequenceid=264242958, filesize=8.3 M 2020-11-17 13:05:27,649 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.42 MB/106349620, heapSize ~128.18 MB/134401848, currentSize=1.65 MB/1724954 for e5dc5c620be19b02666a19c280ff582e in 834ms, sequenceid=264242958, compaction requested=true 2020-11-17 13:05:27,651 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 13:05:27,651 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ec7e959c51a54c389c4a75a346e7ddfe, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6962ad15beae48f18b2278c8874d4ff8, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/2480e1d2210a4362b11c3f7c1de5749d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=111.8 M 2020-11-17 13:05:27,683 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:05:27,685 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:05:27,726 INFO [Thread-4065] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:05:38,732 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 186149 ns. 2020-11-17 13:05:38,732 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 395448 ns. 2020-11-17 13:05:38,732 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 57481 ns. 2020-11-17 13:05:38,813 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=98.50 MB heapSize=128.00 MB 2020-11-17 13:05:38,846 INFO [Thread-4067] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:05:39,785 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.50 MB at sequenceid=260959931 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/61ed2f84eba342e588c193d4332a8fd4 2020-11-17 13:05:39,791 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:05:39,810 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/61ed2f84eba342e588c193d4332a8fd4, entries=408836, sequenceid=260959931, filesize=7.9 M 2020-11-17 13:05:39,811 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~98.50 MB/103280514, heapSize ~128.00 MB/134218152, currentSize=58.90 KB/60316 for 06f7ca9534710e37d90d9f8345864259 in 998ms, sequenceid=260959931, compaction requested=true 2020-11-17 13:05:45,173 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#896 average throughput is 36.10 MB/second, slept 15 time(s) and total slept time is 14056 ms. 1 active operations remaining, total limit is 65.38 MB/second 2020-11-17 13:05:45,225 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into f156bade9807484f8f68bdb495eaee6c(size=61.2 M), total size for store is 33.6 G. This selection was in queue for 0sec, and took 21sec to execute. 2020-11-17 13:05:45,225 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605618324204; duration=21sec 2020-11-17 13:05:57,664 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#898 average throughput is 43.81 MB/second, slept 24 time(s) and total slept time is 13648 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 13:05:57,752 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 1bf4cb5d671b4145a444e7a1c9d7ac0a(size=113.5 M), total size for store is 50.0 G. This selection was in queue for 0sec, and took 30sec to execute. 2020-11-17 13:05:57,752 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=9, startTime=1605618327650; duration=30sec 2020-11-17 13:06:38,785 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 51611 ns. 2020-11-17 13:06:48,016 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:06:59,207 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618318304 with entries=37133, filesize=128.09 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618419172 2020-11-17 13:07:33,975 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.95 MB heapSize=128.00 MB 2020-11-17 13:07:34,030 INFO [Thread-4074] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:07:35,203 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.95 MB at sequenceid=262809464 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/a198d38ec0994a8d9e732e7a41e5476e 2020-11-17 13:07:35,209 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:07:35,230 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/a198d38ec0994a8d9e732e7a41e5476e, entries=575851, sequenceid=262809464, filesize=9.1 M 2020-11-17 13:07:35,232 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.95 MB/90129296, heapSize ~128.00 MB/134219832, currentSize=83.27 KB/85267 for 73aae5715542e1a5370f7540d2117abc in 1257ms, sequenceid=262809464, compaction requested=true 2020-11-17 13:07:35,233 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 13:07:35,233 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/bc2d2247c7234ec6b513d7e6278ced71, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/ad01bb4b93f741eb925cd488c67f00ae, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/a198d38ec0994a8d9e732e7a41e5476e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=27.1 M 2020-11-17 13:07:35,268 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:07:35,274 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:07:35,305 INFO [Thread-4076] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:07:40,004 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#901 average throughput is 53.98 MB/second, slept 3 time(s) and total slept time is 332 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 13:07:40,076 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 68e11a44360849008edc7b5514b55aca(size=27.6 M), total size for store is 12.7 G. This selection was in queue for 0sec, and took 4sec to execute. 2020-11-17 13:07:40,076 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605618455232; duration=4sec 2020-11-17 13:07:48,886 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:08:05,524 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:08:10,427 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:08:38,327 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618419172 with entries=37420, filesize=128.53 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618518280 2020-11-17 13:08:38,671 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 49182 ns. 2020-11-17 13:08:55,090 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.94 MB heapSize=128.02 MB 2020-11-17 13:08:55,123 INFO [Thread-4081] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:08:55,838 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.94 MB at sequenceid=264253497 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/818f384566744b9aa784fb0cf64614b5 2020-11-17 13:08:55,844 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:08:55,860 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/818f384566744b9aa784fb0cf64614b5, entries=347744, sequenceid=264253497, filesize=8.1 M 2020-11-17 13:08:55,861 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.94 MB/107935818, heapSize ~128.02 MB/134243336, currentSize=442.99 KB/453620 for e5dc5c620be19b02666a19c280ff582e in 771ms, sequenceid=264253497, compaction requested=true 2020-11-17 13:08:59,414 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.33 MB heapSize=128.13 MB 2020-11-17 13:08:59,440 INFO [Thread-4083] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:09:00,131 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.33 MB at sequenceid=252527725 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/37683f6e995740e1aeff9237c44ce927 2020-11-17 13:09:00,138 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:09:00,168 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/37683f6e995740e1aeff9237c44ce927, entries=324886, sequenceid=252527725, filesize=6.8 M 2020-11-17 13:09:00,169 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.33 MB/110443794, heapSize ~128.13 MB/134353824, currentSize=21.93 KB/22454 for 82421ce8c5c8b48c573baee83caa8305 in 755ms, sequenceid=252527725, compaction requested=true 2020-11-17 13:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.89 MB, max=1.83 GB, blockCount=44159, accesses=39286229, hits=33524559, hitRatio=85.33%, , cachingAccesses=33576910, cachingHits=33517881, cachingHitsRatio=99.82%, evictions=4919, evicted=0, evictedPerRun=0.0 2020-11-17 13:09:49,053 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 32565 ns. 2020-11-17 13:09:49,053 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 7368 ns. 2020-11-17 13:09:59,773 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:10:11,250 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:10:17,975 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:10:24,474 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:10:36,746 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618518280 with entries=43806, filesize=128.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618636700 2020-11-17 13:11:39,224 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 205764 ns. 2020-11-17 13:11:39,225 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 351346 ns. 2020-11-17 13:11:39,225 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 64563 ns. 2020-11-17 13:11:39,225 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 73820 ns. 2020-11-17 13:11:54,244 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.93 MB heapSize=128.00 MB 2020-11-17 13:11:54,292 INFO [Thread-4092] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:11:55,050 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.93 MB at sequenceid=273044193 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/d11ce318aea947648478de3a53bb08dd 2020-11-17 13:11:55,056 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:11:55,079 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/d11ce318aea947648478de3a53bb08dd, entries=317176, sequenceid=273044193, filesize=8.7 M 2020-11-17 13:11:55,080 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.93 MB/110022669, heapSize ~128.00 MB/134220336, currentSize=30.60 KB/31331 for 0469ce862509756add4a8cdb00bc465c in 836ms, sequenceid=273044193, compaction requested=true 2020-11-17 13:11:55,082 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 13:11:55,082 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/ea598106cac548ec96d3c9d272a5eda1, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/b9ffa0582dda40059fbe057144de5cbc, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/d11ce318aea947648478de3a53bb08dd] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=26.1 M 2020-11-17 13:11:55,112 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:11:55,119 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:11:55,137 INFO [Thread-4094] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:12:00,108 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#905 average throughput is 63.17 MB/second, slept 6 time(s) and total slept time is 2148 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 13:12:00,155 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 0dcce3677198470aa74439a52a9588a9(size=27.0 M), total size for store is 43.0 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 13:12:00,155 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605618715080; duration=5sec 2020-11-17 13:12:15,246 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:12:22,171 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618636700 with entries=39073, filesize=128.64 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618742126 2020-11-17 13:12:32,531 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:13:21,100 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:13:28,793 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:13:31,875 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:13:38,772 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 57801 ns. 2020-11-17 13:13:41,017 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:13:54,413 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.96 MB heapSize=128.28 MB 2020-11-17 13:13:54,454 INFO [Thread-4101] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:13:55,281 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.96 MB at sequenceid=264268491 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/ec9d36f018a34ec9a12dfa29c2069760 2020-11-17 13:13:55,289 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:13:55,307 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ec9d36f018a34ec9a12dfa29c2069760, entries=394338, sequenceid=264268491, filesize=8.1 M 2020-11-17 13:13:55,310 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~99.96 MB/104818211, heapSize ~128.28 MB/134508920, currentSize=429.28 KB/439587 for e5dc5c620be19b02666a19c280ff582e in 897ms, sequenceid=264268491, compaction requested=true 2020-11-17 13:13:59,390 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:14:11,018 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:14:17,741 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.84 MB, max=1.83 GB, blockCount=44152, accesses=39561114, hits=33794339, hitRatio=85.42%, , cachingAccesses=33846703, cachingHits=33787644, cachingHitsRatio=99.83%, evictions=4949, evicted=0, evictedPerRun=0.0 2020-11-17 13:14:36,926 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618742126 with entries=50040, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618876884 2020-11-17 13:14:37,298 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:14:59,239 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 24693 ns. 2020-11-17 13:14:59,239 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 41624 ns. 2020-11-17 13:15:11,244 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:16:10,732 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:16:23,697 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618876884 with entries=41345, filesize=128.38 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618983633 2020-11-17 13:17:38,703 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 194497 ns. 2020-11-17 13:17:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 61833 ns. 2020-11-17 13:17:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 346482 ns. 2020-11-17 13:17:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 58487 ns. 2020-11-17 13:17:39,608 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=101.06 MB heapSize=128.08 MB 2020-11-17 13:17:39,643 INFO [Thread-4112] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:17:40,480 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.06 MB at sequenceid=264280219 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/1458aa68334742a791dfd70cda2cb826 2020-11-17 13:17:40,487 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:17:40,503 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1458aa68334742a791dfd70cda2cb826, entries=374905, sequenceid=264280219, filesize=8.2 M 2020-11-17 13:17:40,506 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.06 MB/105973016, heapSize ~128.08 MB/134304896, currentSize=599.59 KB/613985 for e5dc5c620be19b02666a19c280ff582e in 898ms, sequenceid=264280219, compaction requested=true 2020-11-17 13:17:40,507 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 13:17:40,508 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/818f384566744b9aa784fb0cf64614b5, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/ec9d36f018a34ec9a12dfa29c2069760, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1458aa68334742a791dfd70cda2cb826] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.4 M 2020-11-17 13:17:40,530 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:17:40,533 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:17:40,561 INFO [Thread-4114] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:17:44,869 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618983633 with entries=30348, filesize=128.72 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619064826 2020-11-17 13:17:45,175 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#908 average throughput is 65.56 MB/second, slept 6 time(s) and total slept time is 1271 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 13:17:45,242 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into c2a31be122e0441f8706da1ecfc9d484(size=25.2 M), total size for store is 50.1 G. This selection was in queue for 0sec, and took 4sec to execute. 2020-11-17 13:17:45,242 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=8, startTime=1605619060506; duration=4sec 2020-11-17 13:18:00,803 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:18:17,645 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:18:23,777 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.32 MB heapSize=128.07 MB 2020-11-17 13:18:23,834 INFO [Thread-4118] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:18:24,520 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.32 MB at sequenceid=252544394 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/1795428a6e914f7a8023b74e5937bdf5 2020-11-17 13:18:24,527 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:18:24,544 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/1795428a6e914f7a8023b74e5937bdf5, entries=324024, sequenceid=252544394, filesize=6.6 M 2020-11-17 13:18:24,545 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.32 MB/110438370, heapSize ~128.07 MB/134288288, currentSize=449.95 KB/460749 for 82421ce8c5c8b48c573baee83caa8305 in 768ms, sequenceid=252544394, compaction requested=true 2020-11-17 13:18:24,546 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 13:18:24,546 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/daf25a986df34055806ddbafd51b5cdc, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/37683f6e995740e1aeff9237c44ce927, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/1795428a6e914f7a8023b74e5937bdf5] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=97.2 M 2020-11-17 13:18:24,577 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:18:24,580 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:18:24,601 INFO [Thread-4120] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:18:46,924 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:18:46,926 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:18:47,348 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:18:47,351 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:18:48,805 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:18:49,695 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#910 average throughput is 62.86 MB/second, slept 30 time(s) and total slept time is 10174 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 13:18:49,771 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 7fa784b586c445e7aedfa4c75f5e82d1(size=96.9 M), total size for store is 11.5 G. This selection was in queue for 0sec, and took 25sec to execute. 2020-11-17 13:18:49,771 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605619104545; duration=25sec 2020-11-17 13:18:56,324 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:03,626 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619064826 with entries=28368, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619143574 2020-11-17 13:19:13,314 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=88.74 MB heapSize=128.00 MB 2020-11-17 13:19:13,347 INFO [Thread-4125] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:14,500 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=88.74 MB at sequenceid=292546705 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/ea74fc651b124d40a5e80ff18d0fcc8c 2020-11-17 13:19:14,511 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:14,527 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/ea74fc651b124d40a5e80ff18d0fcc8c, entries=546069, sequenceid=292546705, filesize=9.1 M 2020-11-17 13:19:14,529 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~88.74 MB/93048893, heapSize ~128.00 MB/134218512, currentSize=69.05 KB/70708 for bdaa31a63b7f3d06d9909b7560333e79 in 1215ms, sequenceid=292546705, compaction requested=true 2020-11-17 13:19:14,529 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-17 13:19:14,530 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/96397fb03fa743119f6c3bd7df9e6033, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/5f56686d2a3a4fd4a8833bad3eee8873, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/ea74fc651b124d40a5e80ff18d0fcc8c] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp, totalSize=62.2 M 2020-11-17 13:19:14,573 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:14,577 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:14,603 INFO [Thread-4127] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:19,044 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:20,129 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:20,131 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:22,062 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:22,064 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:22,129 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:22,859 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:22,861 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:23,932 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.07 MB heapSize=128.08 MB 2020-11-17 13:19:23,966 INFO [Thread-4129] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:24,624 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: bdaa31a63b7f3d06d9909b7560333e79#e#compaction#912 average throughput is 59.13 MB/second, slept 7 time(s) and total slept time is 1625 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 13:19:24,704 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of bdaa31a63b7f3d06d9909b7560333e79 into daa59eeafbb74e71884bfee4f286b606(size=61.5 M), total size for store is 31.9 G. This selection was in queue for 0sec, and took 10sec to execute. 2020-11-17 13:19:24,704 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., storeName=e, priority=11, startTime=1605619154529; duration=10sec 2020-11-17 13:19:24,798 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.07 MB at sequenceid=261003001 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/26db2241428543c9b6798fb352d5c30b 2020-11-17 13:19:24,818 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/26db2241428543c9b6798fb352d5c30b, entries=391703, sequenceid=261003001, filesize=7.5 M 2020-11-17 13:19:24,819 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.07 MB/104928232, heapSize ~128.08 MB/134305112, currentSize=235.66 KB/241319 for 06f7ca9534710e37d90d9f8345864259 in 887ms, sequenceid=261003001, compaction requested=true 2020-11-17 13:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=171.39 MB, max=1.83 GB, blockCount=44175, accesses=39607287, hits=33800230, hitRatio=85.34%, , cachingAccesses=33852586, cachingHits=33793504, cachingHitsRatio=99.83%, evictions=4979, evicted=0, evictedPerRun=0.0 2020-11-17 13:19:39,050 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 57958 ns. 2020-11-17 13:19:41,023 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:19:53,248 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:20:08,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15121 ns. 2020-11-17 13:20:08,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 29435 ns. 2020-11-17 13:20:15,681 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619143574 with entries=27765, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619215653 2020-11-17 13:20:15,682 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617072849 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605617072849 2020-11-17 13:20:15,694 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617195219 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605617195219 2020-11-17 13:20:15,700 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617297235 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605617297235 2020-11-17 13:20:15,707 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617419201 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605617419201 2020-11-17 13:20:15,712 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617532446 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605617532446 2020-11-17 13:20:15,718 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617623242 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605617623242 2020-11-17 13:20:15,723 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617754359 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605617754359 2020-11-17 13:20:15,730 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617853701 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605617853701 2020-11-17 13:20:15,739 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605617972719 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605617972719 2020-11-17 13:20:15,744 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618091929 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605618091929 2020-11-17 13:20:41,958 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.65 MB heapSize=128.00 MB 2020-11-17 13:20:41,998 INFO [Thread-4135] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:20:42,647 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.65 MB at sequenceid=273547914 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/f5a69cb8d23f4961b88fb6e162b79bab 2020-11-17 13:20:42,655 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:20:42,673 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/f5a69cb8d23f4961b88fb6e162b79bab, entries=279353, sequenceid=273547914, filesize=8.5 M 2020-11-17 13:20:42,675 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~107.65 MB/112878879, heapSize ~128.00 MB/134219408, currentSize=705.38 KB/722312 for fc93f6dcc564341dcc983bf30971f73d in 717ms, sequenceid=273547914, compaction requested=true 2020-11-17 13:20:58,185 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.78 MB heapSize=128.20 MB 2020-11-17 13:20:58,217 INFO [Thread-4138] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:20:58,899 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.78 MB at sequenceid=252549273 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/780d9bcebe7843fcb85d1d0ccf8881e8 2020-11-17 13:20:58,904 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:20:58,921 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/780d9bcebe7843fcb85d1d0ccf8881e8, entries=306512, sequenceid=252549273, filesize=6.2 M 2020-11-17 13:20:58,922 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.78 MB/111971436, heapSize ~128.20 MB/134432672, currentSize=703.75 KB/720642 for 82421ce8c5c8b48c573baee83caa8305 in 737ms, sequenceid=252549273, compaction requested=true 2020-11-17 13:21:07,489 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619215653 with entries=19300, filesize=128.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619267469 2020-11-17 13:21:09,758 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.15 MB heapSize=128.02 MB 2020-11-17 13:21:09,789 INFO [Thread-4140] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:21:10,518 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.15 MB at sequenceid=264290960 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/34732be4822f44e6b868ca4ad477b642 2020-11-17 13:21:10,524 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:21:10,542 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/34732be4822f44e6b868ca4ad477b642, entries=358074, sequenceid=264290960, filesize=8.0 M 2020-11-17 13:21:10,543 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.15 MB/107114416, heapSize ~128.02 MB/134243376, currentSize=989.51 KB/1013263 for e5dc5c620be19b02666a19c280ff582e in 785ms, sequenceid=264290960, compaction requested=true 2020-11-17 13:21:15,169 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:22:15,635 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619267469 with entries=26278, filesize=128.61 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619335593 2020-11-17 13:22:39,547 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:22:55,344 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=107.09 MB heapSize=128.07 MB 2020-11-17 13:22:55,373 INFO [Thread-4148] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:22:55,888 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.09 MB at sequenceid=252553044 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/e2f5ba5ed20849829abace2a9b72d27d 2020-11-17 13:22:55,894 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:22:55,914 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/e2f5ba5ed20849829abace2a9b72d27d, entries=300721, sequenceid=252553044, filesize=6.2 M 2020-11-17 13:22:55,916 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~107.09 MB/112288642, heapSize ~128.07 MB/134291496, currentSize=326.13 KB/333961 for 82421ce8c5c8b48c573baee83caa8305 in 572ms, sequenceid=252553044, compaction requested=true 2020-11-17 13:22:55,917 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 13:22:55,917 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/7fa784b586c445e7aedfa4c75f5e82d1, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/780d9bcebe7843fcb85d1d0ccf8881e8, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/e2f5ba5ed20849829abace2a9b72d27d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=109.4 M 2020-11-17 13:22:55,949 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:22:55,951 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:22:55,981 INFO [Thread-4150] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:23:16,827 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:23:20,580 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:23:21,477 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:23:21,480 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:23:22,702 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:23:22,704 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:23:23,224 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:23:23,241 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619335593 with entries=25438, filesize=128.10 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619403206 2020-11-17 13:23:23,694 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#918 average throughput is 64.47 MB/second, slept 32 time(s) and total slept time is 11026 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 13:23:23,854 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into a98d81ad0bfd4416a95ce517f596872d(size=109.0 M), total size for store is 11.5 G. This selection was in queue for 0sec, and took 27sec to execute. 2020-11-17 13:23:23,855 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605619375916; duration=27sec 2020-11-17 13:23:37,287 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=86.00 MB heapSize=128.23 MB 2020-11-17 13:23:37,323 INFO [Thread-4154] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:23:38,382 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=86.00 MB at sequenceid=262859589 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/3f7215b87c9c457eb6e894d9ed00e342 2020-11-17 13:23:38,389 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:23:38,406 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/3f7215b87c9c457eb6e894d9ed00e342, entries=574174, sequenceid=262859589, filesize=8.9 M 2020-11-17 13:23:38,408 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~86.00 MB/90179274, heapSize ~128.23 MB/134461232, currentSize=44.42 KB/45491 for 73aae5715542e1a5370f7540d2117abc in 1121ms, sequenceid=262859589, compaction requested=true 2020-11-17 13:23:38,788 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 53790 ns. 2020-11-17 13:23:38,789 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 173784 ns. 2020-11-17 13:23:38,789 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 295946 ns. 2020-11-17 13:23:38,789 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 44644 ns. 2020-11-17 13:23:46,964 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:23:51,125 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:23:58,004 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:24:23,533 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619403206 with entries=22734, filesize=128.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619463492 2020-11-17 13:24:26,950 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:24:33,381 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.85 MB, max=1.83 GB, blockCount=44151, accesses=39643739, hits=33807671, hitRatio=85.28%, , cachingAccesses=33860050, cachingHits=33800942, cachingHitsRatio=99.83%, evictions=5009, evicted=0, evictedPerRun=0.0 2020-11-17 13:25:09,680 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 26307 ns. 2020-11-17 13:25:09,680 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15833 ns. 2020-11-17 13:25:22,349 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.85 MB heapSize=128.01 MB 2020-11-17 13:25:22,385 INFO [Thread-4161] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:25:23,241 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.85 MB at sequenceid=273084113 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/a51e2df3dff04a8c88a212b200f092aa 2020-11-17 13:25:23,249 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:25:23,268 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/a51e2df3dff04a8c88a212b200f092aa, entries=318266, sequenceid=273084113, filesize=8.7 M 2020-11-17 13:25:23,269 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.85 MB/109948056, heapSize ~128.01 MB/134226016, currentSize=43.30 KB/44335 for 0469ce862509756add4a8cdb00bc465c in 920ms, sequenceid=273084113, compaction requested=true 2020-11-17 13:25:29,344 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:25:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 50259 ns. 2020-11-17 13:25:42,046 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:25:49,007 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.00 MB heapSize=128.00 MB 2020-11-17 13:25:49,053 INFO [Thread-4165] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:25:49,948 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.00 MB at sequenceid=264305228 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/552ab48079524ce5bb33ea6da5a3531d 2020-11-17 13:25:50,032 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/552ab48079524ce5bb33ea6da5a3531d, entries=402803, sequenceid=264305228, filesize=8.0 M 2020-11-17 13:25:50,036 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.00 MB/103813802, heapSize ~128.00 MB/134218056, currentSize=206.01 KB/210951 for e5dc5c620be19b02666a19c280ff582e in 1029ms, sequenceid=264305228, compaction requested=true 2020-11-17 13:25:50,038 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 13:25:50,038 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c2a31be122e0441f8706da1ecfc9d484, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/34732be4822f44e6b868ca4ad477b642, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/552ab48079524ce5bb33ea6da5a3531d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.2 M 2020-11-17 13:25:50,064 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:25:50,120 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:25:50,520 INFO [Thread-4167] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:25:50,876 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:25:59,183 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#922 average throughput is 55.85 MB/second, slept 6 time(s) and total slept time is 1247 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 13:25:59,266 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 3d9af353704b473facd9a38e0f45c876(size=42.4 M), total size for store is 50.1 G. This selection was in queue for 0sec, and took 9sec to execute. 2020-11-17 13:25:59,266 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=8, startTime=1605619550036; duration=9sec 2020-11-17 13:26:10,245 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619463492 with entries=38860, filesize=128.17 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619570210 2020-11-17 13:26:21,346 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:26:23,288 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.44 MB heapSize=128.03 MB 2020-11-17 13:26:23,499 INFO [Thread-4170] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:26:24,656 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.44 MB at sequenceid=288746331 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/2c92b9ffb92c4260bdcd266b35263d0c 2020-11-17 13:26:24,665 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:26:24,683 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/2c92b9ffb92c4260bdcd266b35263d0c, entries=525465, sequenceid=288746331, filesize=9.0 M 2020-11-17 13:26:24,685 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.44 MB/94833108, heapSize ~128.04 MB/134254840, currentSize=334.53 KB/342558 for 5e5f05205d7e40b0141cd977dd009ba1 in 1396ms, sequenceid=288746331, compaction requested=true 2020-11-17 13:26:55,435 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=102.53 MB heapSize=128.00 MB 2020-11-17 13:26:55,473 INFO [Thread-4175] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:26:56,159 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.53 MB at sequenceid=261026917 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/92aae0227d8f469b9ed7309d41685cd4 2020-11-17 13:26:56,165 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:26:56,181 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/92aae0227d8f469b9ed7309d41685cd4, entries=358014, sequenceid=261026917, filesize=6.8 M 2020-11-17 13:26:56,183 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.53 MB/107514779, heapSize ~128.00 MB/134217880, currentSize=233.24 KB/238838 for 06f7ca9534710e37d90d9f8345864259 in 748ms, sequenceid=261026917, compaction requested=true 2020-11-17 13:26:56,183 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 13:26:56,184 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/61ed2f84eba342e588c193d4332a8fd4, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/26db2241428543c9b6798fb352d5c30b, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/92aae0227d8f469b9ed7309d41685cd4] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=22.1 M 2020-11-17 13:26:56,214 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:26:56,217 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:26:56,247 INFO [Thread-4177] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:26:56,567 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.90 MB heapSize=128.42 MB 2020-11-17 13:26:56,604 INFO [Thread-4179] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:26:57,235 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.91 MB at sequenceid=252560376 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/0c3702e1f3684fdfaf8387d6d096d78b 2020-11-17 13:26:57,241 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:26:57,257 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0c3702e1f3684fdfaf8387d6d096d78b, entries=308053, sequenceid=252560376, filesize=6.4 M 2020-11-17 13:26:57,258 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.91 MB/112101568, heapSize ~128.44 MB/134678448, currentSize=175.52 KB/179736 for 82421ce8c5c8b48c573baee83caa8305 in 691ms, sequenceid=252560376, compaction requested=true 2020-11-17 13:27:00,797 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#925 average throughput is 65.64 MB/second, slept 5 time(s) and total slept time is 1309 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 13:27:00,868 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into bfc6331a665242cd8f501a9dc0df7beb(size=22.3 M), total size for store is 55.6 G. This selection was in queue for 0sec, and took 4sec to execute. 2020-11-17 13:27:00,868 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605619616183; duration=4sec 2020-11-17 13:27:23,631 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619570210 with entries=27929, filesize=128.49 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619643591 2020-11-17 13:27:23,632 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618181334 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605618181334 2020-11-17 13:27:23,638 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618318304 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605618318304 2020-11-17 13:27:23,645 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618419172 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605618419172 2020-11-17 13:27:23,651 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618518280 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605618518280 2020-11-17 13:27:23,658 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618636700 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605618636700 2020-11-17 13:27:23,664 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618742126 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605618742126 2020-11-17 13:27:23,669 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618876884 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605618876884 2020-11-17 13:27:23,674 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605618983633 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605618983633 2020-11-17 13:27:23,679 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619064826 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605619064826 2020-11-17 13:27:52,089 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:27:55,368 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:28:06,982 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:29:15,803 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619643591 with entries=41354, filesize=128.26 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619755752 2020-11-17 13:29:23,904 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:29:34,669 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.57 MB heapSize=128.12 MB 2020-11-17 13:29:34,708 INFO [Thread-4187] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:29:35,515 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.57 MB at sequenceid=264316637 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/007cb80191dd4abf907fc3e3feaaf258 2020-11-17 13:29:35,521 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:29:35,539 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/007cb80191dd4abf907fc3e3feaaf258, entries=396811, sequenceid=264316637, filesize=8.0 M 2020-11-17 13:29:35,541 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.57 MB/104408743, heapSize ~128.12 MB/134346200, currentSize=353.45 KB/361936 for e5dc5c620be19b02666a19c280ff582e in 872ms, sequenceid=264316637, compaction requested=true 2020-11-17 13:29:36,025 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=172.12 MB, max=1.83 GB, blockCount=44166, accesses=39934228, hits=34085047, hitRatio=85.35%, , cachingAccesses=34137451, cachingHits=34078307, cachingHitsRatio=99.83%, evictions=5039, evicted=0, evictedPerRun=0.0 2020-11-17 13:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 13:29:38,738 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 286730 ns. 2020-11-17 13:29:38,739 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 169482 ns. 2020-11-17 13:29:38,739 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 55478 ns. 2020-11-17 13:29:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 44203 ns. 2020-11-17 13:30:18,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 10587 ns. 2020-11-17 13:30:18,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 32676 ns. 2020-11-17 13:30:38,022 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619755752 with entries=30812, filesize=128.21 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619837992 2020-11-17 13:30:38,817 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 66092 ns. 2020-11-17 13:32:24,531 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619837992 with entries=39732, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619944492 2020-11-17 13:32:36,842 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.42 MB heapSize=128.21 MB 2020-11-17 13:32:36,873 INFO [Thread-4197] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:32:37,441 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.42 MB at sequenceid=252570650 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/7fec4c6dc52b4f2795006f2eb1e06b30 2020-11-17 13:32:37,453 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:32:37,470 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/7fec4c6dc52b4f2795006f2eb1e06b30, entries=311538, sequenceid=252570650, filesize=6.4 M 2020-11-17 13:32:37,472 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.42 MB/111585644, heapSize ~128.21 MB/134434024, currentSize=15.62 KB/15991 for 82421ce8c5c8b48c573baee83caa8305 in 630ms, sequenceid=252570650, compaction requested=true 2020-11-17 13:32:37,473 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 13:32:37,473 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a98d81ad0bfd4416a95ce517f596872d, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/0c3702e1f3684fdfaf8387d6d096d78b, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/7fec4c6dc52b4f2795006f2eb1e06b30] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=121.8 M 2020-11-17 13:32:37,505 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:32:37,507 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:32:37,528 INFO [Thread-4199] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:32:45,831 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:32:52,318 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:33:06,938 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:33:06,940 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:33:07,025 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.27 MB heapSize=128.12 MB 2020-11-17 13:33:07,026 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:33:07,028 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:33:07,062 INFO [Thread-4202] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:33:07,620 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:33:07,922 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.27 MB at sequenceid=264327434 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/0f4bf78cf70142d18ee7caca9e2498e1 2020-11-17 13:33:07,949 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/0f4bf78cf70142d18ee7caca9e2498e1, entries=344135, sequenceid=264327434, filesize=8.1 M 2020-11-17 13:33:07,952 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~103.27 MB/108286749, heapSize ~128.12 MB/134345696, currentSize=1.26 MB/1319391 for e5dc5c620be19b02666a19c280ff582e in 928ms, sequenceid=264327434, compaction requested=true 2020-11-17 13:33:07,953 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 13:33:07,953 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3d9af353704b473facd9a38e0f45c876, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/007cb80191dd4abf907fc3e3feaaf258, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/0f4bf78cf70142d18ee7caca9e2498e1] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=58.5 M 2020-11-17 13:33:07,983 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:33:07,986 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:33:08,016 INFO [Thread-4204] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:33:10,040 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:33:10,616 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#929 average throughput is 60.35 MB/second, slept 36 time(s) and total slept time is 13429 ms. 1 active operations remaining, total limit is 65.38 MB/second 2020-11-17 13:33:10,686 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 43b6bfe7240d4b72ab79dc668ccf478a(size=121.4 M), total size for store is 11.5 G. This selection was in queue for 0sec, and took 33sec to execute. 2020-11-17 13:33:10,686 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=11, startTime=1605619957472; duration=33sec 2020-11-17 13:33:20,321 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#931 average throughput is 57.43 MB/second, slept 14 time(s) and total slept time is 4845 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 13:33:20,434 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 6a857d6869c2455e82b0194be91332d2(size=60.0 M), total size for store is 50.1 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 13:33:20,435 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=8, startTime=1605619987952; duration=12sec 2020-11-17 13:33:39,402 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:33:42,894 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619944492 with entries=28771, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620022845 2020-11-17 13:33:56,762 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:34:07,505 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:34:15,776 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:34:20,536 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:34:30,795 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.67 GB, freeSize=170.12 MB, max=1.83 GB, blockCount=44220, accesses=40326176, hits=34433045, hitRatio=85.39%, , cachingAccesses=34485517, cachingHits=34426295, cachingHitsRatio=99.83%, evictions=5069, evicted=0, evictedPerRun=0.0 2020-11-17 13:34:40,565 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:35:00,255 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:35:18,408 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:35:26,454 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620022845 with entries=38667, filesize=128.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620126410 2020-11-17 13:35:28,818 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 11571 ns. 2020-11-17 13:35:28,818 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 31191 ns. 2020-11-17 13:35:30,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:35:30,433 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:35:30,434 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:35:30,435 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:35:30,500 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:35:38,906 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 279344 ns. 2020-11-17 13:35:38,906 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 180194 ns. 2020-11-17 13:35:38,906 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 50558 ns. 2020-11-17 13:35:38,906 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 43679 ns. 2020-11-17 13:35:38,906 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 41808 ns. 2020-11-17 13:35:46,675 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:36:03,122 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=108.19 MB heapSize=128.65 MB 2020-11-17 13:36:03,169 INFO [Thread-4212] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:36:03,974 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=108.19 MB at sequenceid=273576498 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/942de86a25c147f5a4db87e0b79ab976 2020-11-17 13:36:03,981 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:36:04,000 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/942de86a25c147f5a4db87e0b79ab976, entries=281041, sequenceid=273576498, filesize=8.5 M 2020-11-17 13:36:04,007 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~108.19 MB/113443366, heapSize ~128.65 MB/134899360, currentSize=231.46 KB/237020 for fc93f6dcc564341dcc983bf30971f73d in 885ms, sequenceid=273576498, compaction requested=true 2020-11-17 13:36:04,008 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 13:36:04,008 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/f156bade9807484f8f68bdb495eaee6c, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/f5a69cb8d23f4961b88fb6e162b79bab, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/942de86a25c147f5a4db87e0b79ab976] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=78.1 M 2020-11-17 13:36:04,043 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:36:04,047 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:36:04,070 INFO [Thread-4214] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:36:04,120 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:36:09,121 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:36:16,645 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:36:16,647 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:36:19,698 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#933 average throughput is 61.99 MB/second, slept 19 time(s) and total slept time is 7538 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 13:36:19,817 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into 7420b91c969040b5b09a6eed88318cef(size=78.6 M), total size for store is 33.6 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 13:36:19,817 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605620164007; duration=15sec 2020-11-17 13:36:39,441 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620126410 with entries=26593, filesize=128.35 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620199396 2020-11-17 13:37:09,103 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:09,394 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:09,395 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:09,395 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:09,395 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:09,401 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:09,401 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:09,489 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:09,489 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:09,498 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:09,500 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:26,867 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.40 MB heapSize=128.17 MB 2020-11-17 13:37:26,915 INFO [Thread-4221] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:27,607 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.40 MB at sequenceid=252579271 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/88dca552a79341ed89e5999590b28701 2020-11-17 13:37:27,613 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:27,629 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/88dca552a79341ed89e5999590b28701, entries=311228, sequenceid=252579271, filesize=6.4 M 2020-11-17 13:37:27,632 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.40 MB/111570740, heapSize ~128.17 MB/134394480, currentSize=58.72 KB/60129 for 82421ce8c5c8b48c573baee83caa8305 in 766ms, sequenceid=252579271, compaction requested=true 2020-11-17 13:37:32,690 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:54,195 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.27 MB heapSize=128.05 MB 2020-11-17 13:37:54,246 INFO [Thread-4224] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:55,086 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.27 MB at sequenceid=261061153 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/217d88d91a574e1196460e36cd1d28a4 2020-11-17 13:37:55,093 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:55,108 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/217d88d91a574e1196460e36cd1d28a4, entries=374066, sequenceid=261061153, filesize=7.1 M 2020-11-17 13:37:55,110 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.27 MB/106194117, heapSize ~128.05 MB/134269280, currentSize=247.45 KB/253390 for 06f7ca9534710e37d90d9f8345864259 in 914ms, sequenceid=261061153, compaction requested=true 2020-11-17 13:37:58,569 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.53 MB heapSize=128.09 MB 2020-11-17 13:37:58,593 INFO [Thread-4226] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:59,628 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.53 MB at sequenceid=264342015 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/5916400b82a24eea92b99f0b23452701 2020-11-17 13:37:59,634 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:37:59,654 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5916400b82a24eea92b99f0b23452701, entries=397066, sequenceid=264342015, filesize=8.1 M 2020-11-17 13:37:59,655 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.53 MB/104363988, heapSize ~128.09 MB/134314312, currentSize=214.78 KB/219938 for e5dc5c620be19b02666a19c280ff582e in 1086ms, sequenceid=264342015, compaction requested=true 2020-11-17 13:38:22,555 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620199396 with entries=38136, filesize=128.17 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620302506 2020-11-17 13:38:57,072 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.96 MB heapSize=128.00 MB 2020-11-17 13:38:57,102 INFO [Thread-4231] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:38:57,852 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.96 MB at sequenceid=273123403 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/c5bbdf936a7240ffa176e035347cb50d 2020-11-17 13:38:57,857 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:38:57,875 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/c5bbdf936a7240ffa176e035347cb50d, entries=316690, sequenceid=273123403, filesize=8.7 M 2020-11-17 13:38:57,877 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.96 MB/110059374, heapSize ~128.00 MB/134218656, currentSize=20.28 KB/20767 for 0469ce862509756add4a8cdb00bc465c in 804ms, sequenceid=273123403, compaction requested=true 2020-11-17 13:38:57,878 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 13:38:57,878 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/0dcce3677198470aa74439a52a9588a9, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/a51e2df3dff04a8c88a212b200f092aa, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/c5bbdf936a7240ffa176e035347cb50d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=44.3 M 2020-11-17 13:38:57,922 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:38:57,923 INFO [Thread-4233] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:38:57,926 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:38:58,402 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:39:06,275 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#938 average throughput is 62.62 MB/second, slept 8 time(s) and total slept time is 2501 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 13:39:06,446 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 5df77665faae42c2b51add5e51f44a59(size=45.1 M), total size for store is 43.0 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 13:39:06,446 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605620337877; duration=8sec 2020-11-17 13:39:21,256 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.81 MB heapSize=128.00 MB 2020-11-17 13:39:21,292 INFO [Thread-4236] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:39:22,385 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.81 MB at sequenceid=262908423 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/a743c25af8914c298d17781bcef2a248 2020-11-17 13:39:22,391 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:39:22,407 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/a743c25af8914c298d17781bcef2a248, entries=577903, sequenceid=262908423, filesize=9.0 M 2020-11-17 13:39:22,408 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.81 MB/89977219, heapSize ~128.00 MB/134218064, currentSize=72.50 KB/74243 for 73aae5715542e1a5370f7540d2117abc in 1152ms, sequenceid=262908423, compaction requested=true 2020-11-17 13:39:22,409 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 13:39:22,409 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/68e11a44360849008edc7b5514b55aca, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/3f7215b87c9c457eb6e894d9ed00e342, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/a743c25af8914c298d17781bcef2a248] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=45.4 M 2020-11-17 13:39:22,452 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:39:22,459 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:39:22,474 INFO [Thread-4238] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:39:27,653 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:39:28,568 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:39:29,158 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#940 average throughput is 62.80 MB/second, slept 4 time(s) and total slept time is 472 ms. 0 active operations remaining, total limit is 65.38 MB/second 2020-11-17 13:39:29,232 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into a3dcbdf825d54d279f0eda98398d5efb(size=46.0 M), total size for store is 12.8 G. This selection was in queue for 0sec, and took 6sec to execute. 2020-11-17 13:39:29,232 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605620362408; duration=6sec 2020-11-17 13:39:34,209 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620302506 with entries=26478, filesize=128.42 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620374181 2020-11-17 13:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.69 GB, freeSize=147.36 MB, max=1.83 GB, blockCount=44715, accesses=40921122, hits=34996414, hitRatio=85.52%, , cachingAccesses=35049372, cachingHits=34989593, cachingHitsRatio=99.83%, evictions=5099, evicted=0, evictedPerRun=0.0 2020-11-17 13:40:13,739 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:40:38,788 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14154 ns. 2020-11-17 13:40:38,788 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 48392 ns. 2020-11-17 13:40:57,009 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:04,393 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:10,110 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:13,781 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:20,353 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620374181 with entries=39086, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620480303 2020-11-17 13:41:34,153 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=99.45 MB heapSize=128.47 MB 2020-11-17 13:41:34,200 INFO [Thread-4245] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:35,255 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=99.52 MB at sequenceid=264352953 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/29687b71ef40469eb91e0e5ee550778e 2020-11-17 13:41:35,262 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:35,282 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/29687b71ef40469eb91e0e5ee550778e, entries=403534, sequenceid=264352953, filesize=8.1 M 2020-11-17 13:41:35,284 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~99.52 MB/104356368, heapSize ~128.56 MB/134809272, currentSize=1.65 MB/1729176 for e5dc5c620be19b02666a19c280ff582e in 1132ms, sequenceid=264352953, compaction requested=true 2020-11-17 13:41:35,285 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 13:41:35,285 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/fb7c10a8859e4388997667a45930d8a3, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/57c9b765dec642ad8acdfdc15f3eb4b5, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/3183b857c28f4c2badb7959ed3ec1687, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/1bf4cb5d671b4145a444e7a1c9d7ac0a, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6a857d6869c2455e82b0194be91332d2, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5916400b82a24eea92b99f0b23452701, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/29687b71ef40469eb91e0e5ee550778e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=2.5 G 2020-11-17 13:41:35,412 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:35,429 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:35,436 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:35,454 INFO [Thread-4247] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:35,486 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:35,538 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:35,543 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:37,043 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:37,116 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:38,715 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 285476 ns. 2020-11-17 13:41:38,716 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 41032 ns. 2020-11-17 13:41:38,716 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 153083 ns. 2020-11-17 13:41:38,716 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 53766 ns. 2020-11-17 13:41:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 33785 ns. 2020-11-17 13:41:48,503 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:41:56,524 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:42:15,248 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:42:28,629 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.21 MB heapSize=128.03 MB 2020-11-17 13:42:28,663 INFO [Thread-4255] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:42:29,355 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.21 MB at sequenceid=252588157 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/026562c7cf0d46398ba5cdafd6308f53 2020-11-17 13:42:29,362 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:42:29,380 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/026562c7cf0d46398ba5cdafd6308f53, entries=311821, sequenceid=252588157, filesize=6.3 M 2020-11-17 13:42:29,382 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.21 MB/111368919, heapSize ~128.03 MB/134246376, currentSize=313.11 KB/320626 for 82421ce8c5c8b48c573baee83caa8305 in 753ms, sequenceid=252588157, compaction requested=true 2020-11-17 13:42:32,097 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620480303 with entries=26306, filesize=128.18 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620552075 2020-11-17 13:42:37,088 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:42:56,897 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:43:00,850 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:43:02,496 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:43:06,162 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:43:20,865 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:43:24,277 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:43:46,819 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:44:08,719 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:44:09,217 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620552075 with entries=35274, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620649177 2020-11-17 13:44:10,619 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:44:30,279 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.69 GB, freeSize=148.87 MB, max=1.83 GB, blockCount=44664, accesses=41449108, hits=35332515, hitRatio=85.24%, , cachingAccesses=35385429, cachingHits=35325624, cachingHitsRatio=99.83%, evictions=5129, evicted=0, evictedPerRun=0.0 2020-11-17 13:44:52,174 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:44:58,131 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:45:09,020 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.89 MB heapSize=128.40 MB 2020-11-17 13:45:09,065 INFO [Thread-4271] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:45:09,984 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.89 MB at sequenceid=264363713 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/d2159986a64e4b589531390110154bac 2020-11-17 13:45:09,991 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:45:10,019 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d2159986a64e4b589531390110154bac, entries=353413, sequenceid=264363713, filesize=8.1 M 2020-11-17 13:45:10,021 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.89 MB/107887896, heapSize ~128.40 MB/134633344, currentSize=948.08 KB/970833 for e5dc5c620be19b02666a19c280ff582e in 1002ms, sequenceid=264363713, compaction requested=false 2020-11-17 13:45:13,139 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:45:13,721 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:45:14,904 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620649177 with entries=25012, filesize=128.08 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620714886 2020-11-17 13:45:16,993 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:45:35,263 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:45:38,854 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 20886 ns. 2020-11-17 13:45:38,854 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 27476 ns. 2020-11-17 13:45:55,981 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:00,954 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:05,312 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.68 MB heapSize=128.22 MB 2020-11-17 13:46:05,343 INFO [Thread-4278] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:05,912 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.68 MB at sequenceid=252594525 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/aa43f495834546608bb8da2d01c75b8f 2020-11-17 13:46:05,918 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:05,936 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/aa43f495834546608bb8da2d01c75b8f, entries=308064, sequenceid=252594525, filesize=6.3 M 2020-11-17 13:46:05,937 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.68 MB/111858030, heapSize ~128.22 MB/134443856, currentSize=14.02 KB/14356 for 82421ce8c5c8b48c573baee83caa8305 in 626ms, sequenceid=252594525, compaction requested=true 2020-11-17 13:46:05,938 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 13:46:05,939 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/88dca552a79341ed89e5999590b28701, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/026562c7cf0d46398ba5cdafd6308f53, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/aa43f495834546608bb8da2d01c75b8f] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=19.0 M 2020-11-17 13:46:05,968 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:05,971 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:05,999 INFO [Thread-4280] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:14,447 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#946 average throughput is 37.69 MB/second, slept 6 time(s) and total slept time is 5304 ms. 1 active operations remaining, total limit is 73.08 MB/second 2020-11-17 13:46:14,509 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 82a200f5d39b41e799d5d480f95d4b74(size=18.9 M), total size for store is 11.5 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 13:46:14,509 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605620765937; duration=8sec 2020-11-17 13:46:20,651 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:21,322 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:30,813 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:32,781 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:37,425 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:37,427 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 44488 ns. 2020-11-17 13:46:38,860 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:38,969 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:39,452 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:42,518 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620714886 with entries=31952, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620802496 2020-11-17 13:46:44,499 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:46,310 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.86 MB heapSize=128.02 MB 2020-11-17 13:46:46,341 INFO [Thread-4286] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:47,126 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.86 MB at sequenceid=261088707 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/42457eebee664653b4bb35032d4e0cab 2020-11-17 13:46:47,149 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/42457eebee664653b4bb35032d4e0cab, entries=364968, sequenceid=261088707, filesize=6.9 M 2020-11-17 13:46:47,150 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.86 MB/106811198, heapSize ~128.02 MB/134240528, currentSize=164.10 KB/168036 for 06f7ca9534710e37d90d9f8345864259 in 840ms, sequenceid=261088707, compaction requested=true 2020-11-17 13:46:47,151 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 13:46:47,151 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/bfc6331a665242cd8f501a9dc0df7beb, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/217d88d91a574e1196460e36cd1d28a4, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/42457eebee664653b4bb35032d4e0cab] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=36.3 M 2020-11-17 13:46:47,184 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:47,194 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:47,214 INFO [Thread-4288] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:46:48,063 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:47:00,911 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#948 average throughput is 36.61 MB/second, slept 10 time(s) and total slept time is 8579 ms. 1 active operations remaining, total limit is 73.08 MB/second 2020-11-17 13:47:01,009 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 82759d654574476693ae34731187c6a5(size=36.5 M), total size for store is 55.7 G. This selection was in queue for 0sec, and took 13sec to execute. 2020-11-17 13:47:01,009 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605620807151; duration=13sec 2020-11-17 13:47:05,892 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:47:07,821 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:47:11,032 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:47:23,458 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=87.41 MB heapSize=128.01 MB 2020-11-17 13:47:23,506 INFO [Thread-4293] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:47:24,872 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=87.41 MB at sequenceid=292635570 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/dad1b23bae454ef0ac01387a693f2143 2020-11-17 13:47:24,884 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:47:24,918 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/dad1b23bae454ef0ac01387a693f2143, entries=563204, sequenceid=292635570, filesize=8.5 M 2020-11-17 13:47:24,919 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~87.41 MB/91654486, heapSize ~128.01 MB/134231560, currentSize=76.07 KB/77895 for bdaa31a63b7f3d06d9909b7560333e79 in 1461ms, sequenceid=292635570, compaction requested=true 2020-11-17 13:47:31,230 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:47:39,130 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 42445 ns. 2020-11-17 13:47:39,131 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 313933 ns. 2020-11-17 13:47:39,131 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 169844 ns. 2020-11-17 13:47:39,131 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 57389 ns. 2020-11-17 13:47:52,697 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:47:54,280 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:47:56,106 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620802496 with entries=24910, filesize=128.39 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620876073 2020-11-17 13:47:56,107 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619143574 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605619143574 2020-11-17 13:47:56,120 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619215653 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605619215653 2020-11-17 13:47:56,127 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619267469 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605619267469 2020-11-17 13:47:56,136 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619335593 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605619335593 2020-11-17 13:47:56,142 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619403206 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605619403206 2020-11-17 13:47:56,149 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619463492 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605619463492 2020-11-17 13:48:17,015 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:25,773 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:25,775 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:25,854 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:26,932 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:27,864 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:28,124 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:28,211 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:28,527 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:28,599 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:28,690 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:29,382 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:29,384 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:29,509 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:29,511 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:29,518 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:32,120 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:32,122 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:32,877 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:33,080 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:33,082 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:34,343 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:34,348 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:34,350 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:34,352 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:34,354 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:34,440 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:34,442 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:34,445 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:35,083 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:35,085 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:35,873 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:36,071 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:36,074 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:36,093 INFO [DataStreamer for file /hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/a553cd49816f4f9b9f3bc94c3c3cc20d] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:36,648 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:37,191 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:37,386 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:37,388 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:37,912 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:38,092 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:38,635 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:38,637 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:38,828 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:39,400 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:39,942 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:39,944 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:40,120 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:40,675 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:40,842 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:41,401 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:41,404 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:41,406 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:42,189 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:42,361 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:42,363 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:42,932 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:43,479 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:43,659 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:44,217 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:44,219 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:44,422 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:45,064 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:45,239 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:45,241 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:46,489 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:46,491 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:47,009 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:47,153 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:47,677 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:47,681 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:47,822 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:48,349 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:48,497 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:48,500 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:49,032 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:49,216 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:49,691 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:49,693 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:49,840 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:50,369 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:50,624 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:50,629 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:50,631 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:50,633 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:50,830 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:50,834 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:50,836 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:51,289 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620876073 with entries=20272, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620931260 2020-11-17 13:48:52,576 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:52,579 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:52,691 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:52,806 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:52,891 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:53,184 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:53,186 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:53,261 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:55,868 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#942 average throughput is 66.29 MB/second, slept 509 time(s) and total slept time is 135000 ms. 0 active operations remaining, total limit is 73.08 MB/second 2020-11-17 13:48:55,918 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:48:55,944 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 7 file(s) in e of e5dc5c620be19b02666a19c280ff582e into a553cd49816f4f9b9f3bc94c3c3cc20d(size=2.5 G), total size for store is 50.1 G. This selection was in queue for 0sec, and took 7mins, 20sec to execute. 2020-11-17 13:48:55,944 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=8, startTime=1605620495284; duration=7mins, 20sec 2020-11-17 13:48:59,776 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.88 MB heapSize=128.00 MB 2020-11-17 13:48:59,802 INFO [Thread-4303] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:49:00,662 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.88 MB at sequenceid=264375214 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/87ed54e1ca9c4a779d99219efc260b27 2020-11-17 13:49:00,671 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:49:00,691 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/87ed54e1ca9c4a779d99219efc260b27, entries=375847, sequenceid=264375214, filesize=8.1 M 2020-11-17 13:49:00,696 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.88 MB/105779116, heapSize ~128.00 MB/134218400, currentSize=25.79 KB/26406 for e5dc5c620be19b02666a19c280ff582e in 920ms, sequenceid=264375214, compaction requested=true 2020-11-17 13:49:06,339 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.66 MB heapSize=128.21 MB 2020-11-17 13:49:06,359 INFO [Thread-4305] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:49:06,875 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.66 MB at sequenceid=252599743 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/dce15dacf30a480dbfa28479126c85f4 2020-11-17 13:49:06,882 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:49:06,905 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/dce15dacf30a480dbfa28479126c85f4, entries=308173, sequenceid=252599743, filesize=6.2 M 2020-11-17 13:49:06,906 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.66 MB/111841661, heapSize ~128.21 MB/134435872, currentSize=24.18 KB/24757 for 82421ce8c5c8b48c573baee83caa8305 in 567ms, sequenceid=252599743, compaction requested=true 2020-11-17 13:49:24,099 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=90.25 MB heapSize=128.01 MB 2020-11-17 13:49:24,132 INFO [Thread-4308] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:49:25,086 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=90.25 MB at sequenceid=288821239 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/e86ce2d667d04ff1bbb54a5675e125a4 2020-11-17 13:49:25,092 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:49:25,109 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/e86ce2d667d04ff1bbb54a5675e125a4, entries=527798, sequenceid=288821239, filesize=9.0 M 2020-11-17 13:49:25,113 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~90.25 MB/94636960, heapSize ~128.01 MB/134226384, currentSize=48.64 KB/49811 for 5e5f05205d7e40b0141cd977dd009ba1 in 1014ms, sequenceid=288821239, compaction requested=true 2020-11-17 13:49:25,114 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 13:49:25,114 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/745d0cf0e6a743dcbbfadc5a8100f779, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/2c92b9ffb92c4260bdcd266b35263d0c, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/e86ce2d667d04ff1bbb54a5675e125a4] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=45.5 M 2020-11-17 13:49:25,148 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:49:25,155 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:49:25,169 INFO [Thread-4310] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:49:30,232 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:49:31,313 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:49:31,644 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#953 average throughput is 68.40 MB/second, slept 6 time(s) and total slept time is 970 ms. 0 active operations remaining, total limit is 73.08 MB/second 2020-11-17 13:49:31,704 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into f7a2afe80a8746008d7160e8a7b397ad(size=45.8 M), total size for store is 24.5 G. This selection was in queue for 0sec, and took 6sec to execute. 2020-11-17 13:49:31,704 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=10, startTime=1605620965113; duration=6sec 2020-11-17 13:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.69 GB, freeSize=146.00 MB, max=1.83 GB, blockCount=44692, accesses=41758239, hits=35339010, hitRatio=84.63%, , cachingAccesses=35391689, cachingHits=35331827, cachingHitsRatio=99.83%, evictions=5159, evicted=0, evictedPerRun=0.0 2020-11-17 13:49:50,943 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:50:06,640 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620931260 with entries=28349, filesize=128.08 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621006611 2020-11-17 13:50:06,641 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619570210 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605619570210 2020-11-17 13:50:06,652 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619643591 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605619643591 2020-11-17 13:50:06,659 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619755752 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605619755752 2020-11-17 13:50:06,664 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619837992 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605619837992 2020-11-17 13:50:06,670 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605619944492 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605619944492 2020-11-17 13:50:06,676 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620022845 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605620022845 2020-11-17 13:50:40,991 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:50:48,880 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 17890 ns. 2020-11-17 13:50:48,880 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 29104 ns. 2020-11-17 13:50:58,827 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621006611 with entries=19767, filesize=128.09 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621058776 2020-11-17 13:51:18,558 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.97 MB heapSize=128.52 MB 2020-11-17 13:51:18,614 INFO [Thread-4316] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:51:19,471 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.97 MB at sequenceid=273604148 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/f39b5cc8518e407d8e3aa08ca8029744 2020-11-17 13:51:19,476 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:51:19,494 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/f39b5cc8518e407d8e3aa08ca8029744, entries=282487, sequenceid=273604148, filesize=8.4 M 2020-11-17 13:51:19,496 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~107.97 MB/113212546, heapSize ~128.52 MB/134766768, currentSize=56.29 KB/57638 for fc93f6dcc564341dcc983bf30971f73d in 938ms, sequenceid=273604148, compaction requested=true 2020-11-17 13:51:23,146 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.85 MB heapSize=128.27 MB 2020-11-17 13:51:23,191 INFO [Thread-4318] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:51:23,818 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.85 MB at sequenceid=252604063 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/dec567534adb45f8aa5ea2aa33102d25 2020-11-17 13:51:23,824 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:51:23,841 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/dec567534adb45f8aa5ea2aa33102d25, entries=306551, sequenceid=252604063, filesize=6.2 M 2020-11-17 13:51:23,843 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.85 MB/112043623, heapSize ~128.27 MB/134505368, currentSize=414.33 KB/424269 for 82421ce8c5c8b48c573baee83caa8305 in 697ms, sequenceid=252604063, compaction requested=true 2020-11-17 13:51:23,843 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 13:51:23,844 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/82a200f5d39b41e799d5d480f95d4b74, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/dce15dacf30a480dbfa28479126c85f4, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/dec567534adb45f8aa5ea2aa33102d25] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=31.3 M 2020-11-17 13:51:23,880 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:51:23,882 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:51:23,908 INFO [Thread-4320] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:51:32,964 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#956 average throughput is 58.70 MB/second, slept 10 time(s) and total slept time is 3961 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 13:51:33,092 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 5ffc3511d0fe4c13b7bb10a2964a2a93(size=31.2 M), total size for store is 11.6 G. This selection was in queue for 0sec, and took 9sec to execute. 2020-11-17 13:51:33,092 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605621083843; duration=9sec 2020-11-17 13:51:38,906 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 46005 ns. 2020-11-17 13:51:54,120 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621058776 with entries=20751, filesize=128.18 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621114084 2020-11-17 13:51:54,121 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620126410 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605620126410 2020-11-17 13:51:54,133 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620199396 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605620199396 2020-11-17 13:52:35,555 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=102.81 MB heapSize=128.04 MB 2020-11-17 13:52:35,601 INFO [Thread-4326] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:52:36,314 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.81 MB at sequenceid=261106892 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/6150ad1f6f2e481fa2cf84bc887f8b21 2020-11-17 13:52:36,321 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:52:36,343 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/6150ad1f6f2e481fa2cf84bc887f8b21, entries=352822, sequenceid=261106892, filesize=6.6 M 2020-11-17 13:52:36,346 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.81 MB/107804045, heapSize ~128.04 MB/134260048, currentSize=290.34 KB/297307 for 06f7ca9534710e37d90d9f8345864259 in 792ms, sequenceid=261106892, compaction requested=true 2020-11-17 13:52:39,555 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.16 MB heapSize=128.35 MB 2020-11-17 13:52:39,587 INFO [Thread-4328] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:52:40,349 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.16 MB at sequenceid=273163023 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/e6d6d08014594692a937b585b34ee501 2020-11-17 13:52:40,355 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:52:40,370 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/e6d6d08014594692a937b585b34ee501, entries=318914, sequenceid=273163023, filesize=8.7 M 2020-11-17 13:52:40,371 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.16 MB/110267568, heapSize ~128.35 MB/134589080, currentSize=23.89 KB/24466 for 0469ce862509756add4a8cdb00bc465c in 816ms, sequenceid=273163023, compaction requested=true 2020-11-17 13:53:07,542 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621114084 with entries=27327, filesize=128.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621187502 2020-11-17 13:53:10,425 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=97.97 MB heapSize=128.07 MB 2020-11-17 13:53:10,454 INFO [Thread-4331] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:53:11,324 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.97 MB at sequenceid=264388130 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/c218ef97afab4e63bf9ff61aabf4fcd7 2020-11-17 13:53:11,330 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:53:11,349 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c218ef97afab4e63bf9ff61aabf4fcd7, entries=418136, sequenceid=264388130, filesize=7.9 M 2020-11-17 13:53:11,351 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~97.97 MB/102732021, heapSize ~128.07 MB/134288816, currentSize=439.51 KB/450063 for e5dc5c620be19b02666a19c280ff582e in 926ms, sequenceid=264388130, compaction requested=true 2020-11-17 13:53:11,352 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 13:53:11,352 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d2159986a64e4b589531390110154bac, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/87ed54e1ca9c4a779d99219efc260b27, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c218ef97afab4e63bf9ff61aabf4fcd7] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=24.1 M 2020-11-17 13:53:11,384 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:53:11,387 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:53:11,411 INFO [Thread-4333] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:53:16,619 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#960 average throughput is 57.74 MB/second, slept 6 time(s) and total slept time is 1793 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 13:53:16,670 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 7e735b101c144c8d87efa654475d63e9(size=24.9 M), total size for store is 50.1 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 13:53:16,670 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605621191351; duration=5sec 2020-11-17 13:53:33,552 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=107.36 MB heapSize=128.37 MB 2020-11-17 13:53:33,584 INFO [Thread-4336] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:53:34,178 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.36 MB at sequenceid=252608157 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/730323a4112d4bad88a324b01b75db73 2020-11-17 13:53:34,184 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:53:34,202 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/730323a4112d4bad88a324b01b75db73, entries=301140, sequenceid=252608157, filesize=6.2 M 2020-11-17 13:53:34,204 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~107.36 MB/112578822, heapSize ~128.37 MB/134609712, currentSize=26.05 KB/26679 for 82421ce8c5c8b48c573baee83caa8305 in 653ms, sequenceid=252608157, compaction requested=true 2020-11-17 13:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 56139 ns. 2020-11-17 13:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 55216 ns. 2020-11-17 13:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 179590 ns. 2020-11-17 13:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 372389 ns. 2020-11-17 13:54:07,261 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621187502 with entries=22641, filesize=128.52 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621247201 2020-11-17 13:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.68 GB, freeSize=159.38 MB, max=1.83 GB, blockCount=44328, accesses=41774232, hits=35341441, hitRatio=84.60%, , cachingAccesses=35394150, cachingHits=35334257, cachingHitsRatio=99.83%, evictions=5189, evicted=0, evictedPerRun=0.0 2020-11-17 13:54:40,416 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.63 MB heapSize=128.15 MB 2020-11-17 13:54:40,460 INFO [Thread-4340] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:54:41,933 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.63 MB at sequenceid=262955777 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/579008574a9e4a719941ec0c8ad61005 2020-11-17 13:54:41,939 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:54:41,959 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/579008574a9e4a719941ec0c8ad61005, entries=579278, sequenceid=262955777, filesize=8.9 M 2020-11-17 13:54:41,960 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.63 MB/89785515, heapSize ~128.15 MB/134372592, currentSize=85.79 KB/87849 for 73aae5715542e1a5370f7540d2117abc in 1544ms, sequenceid=262955777, compaction requested=true 2020-11-17 13:55:21,542 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:55:41,941 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621247201 with entries=35591, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621341908 2020-11-17 13:55:41,941 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620302506 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605620302506 2020-11-17 13:55:41,947 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620374181 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605620374181 2020-11-17 13:55:41,954 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620480303 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605620480303 2020-11-17 13:55:41,960 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620552075 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605620552075 2020-11-17 13:55:41,966 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620649177 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605620649177 2020-11-17 13:55:41,972 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620714886 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605620714886 2020-11-17 13:55:59,297 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 17380 ns. 2020-11-17 13:55:59,297 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 101962 ns. 2020-11-17 13:56:18,300 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:56:27,600 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:56:47,795 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.41 MB heapSize=128.32 MB 2020-11-17 13:56:47,851 INFO [Thread-4347] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:56:48,678 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.41 MB at sequenceid=264399279 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/5dbc2491b78345459cea98e9fd2163fe 2020-11-17 13:56:48,683 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:56:48,704 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5dbc2491b78345459cea98e9fd2163fe, entries=387513, sequenceid=264399279, filesize=8.2 M 2020-11-17 13:56:48,705 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.41 MB/105283514, heapSize ~128.32 MB/134556384, currentSize=1.01 MB/1054677 for e5dc5c620be19b02666a19c280ff582e in 910ms, sequenceid=264399279, compaction requested=true 2020-11-17 13:57:01,292 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:57:05,155 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621341908 with entries=30664, filesize=128.14 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621425115 2020-11-17 13:57:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 48142 ns. 2020-11-17 13:57:39,626 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=107.18 MB heapSize=128.55 MB 2020-11-17 13:57:39,681 INFO [Thread-4351] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:57:40,405 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.18 MB at sequenceid=252615618 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/fc90b98709a64a9dbbe4f36c08708c51 2020-11-17 13:57:40,411 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:57:40,426 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/fc90b98709a64a9dbbe4f36c08708c51, entries=305946, sequenceid=252615618, filesize=6.4 M 2020-11-17 13:57:40,427 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~107.18 MB/112390881, heapSize ~128.55 MB/134791216, currentSize=657.68 KB/673464 for 82421ce8c5c8b48c573baee83caa8305 in 801ms, sequenceid=252615618, compaction requested=true 2020-11-17 13:57:40,428 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 13:57:40,428 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/5ffc3511d0fe4c13b7bb10a2964a2a93, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/730323a4112d4bad88a324b01b75db73, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/fc90b98709a64a9dbbe4f36c08708c51] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=43.8 M 2020-11-17 13:57:40,457 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:57:40,459 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:57:40,484 INFO [Thread-4353] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:57:53,758 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#965 average throughput is 56.20 MB/second, slept 12 time(s) and total slept time is 5088 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 13:57:53,816 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 6c412cb04d694736a0a3e8fe9446bb62(size=43.7 M), total size for store is 11.6 G. This selection was in queue for 0sec, and took 13sec to execute. 2020-11-17 13:57:53,816 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605621460427; duration=13sec 2020-11-17 13:57:59,872 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:58:36,843 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 13:58:38,731 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 50753 ns. 2020-11-17 13:58:44,692 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621425115 with entries=37515, filesize=128.18 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621524658 2020-11-17 13:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.68 GB, freeSize=158.03 MB, max=1.83 GB, blockCount=44354, accesses=42058997, hits=35614077, hitRatio=84.68%, , cachingAccesses=35666812, cachingHits=35606893, cachingHitsRatio=99.83%, evictions=5219, evicted=0, evictedPerRun=0.0 2020-11-17 13:59:39,287 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 189726 ns. 2020-11-17 13:59:39,288 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 396116 ns. 2020-11-17 13:59:39,288 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 71079 ns. 2020-11-17 14:00:06,759 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621524658 with entries=30328, filesize=128.30 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621606715 2020-11-17 14:00:21,626 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.68 MB heapSize=128.00 MB 2020-11-17 14:00:21,678 INFO [Thread-4362] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:00:22,601 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.68 MB at sequenceid=264410324 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/e110dc08d5a74ddda8eb097fbe93652d 2020-11-17 14:00:22,608 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:00:22,626 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e110dc08d5a74ddda8eb097fbe93652d, entries=350987, sequenceid=264410324, filesize=8.1 M 2020-11-17 14:00:22,628 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.68 MB/107668178, heapSize ~128.00 MB/134220264, currentSize=1.06 MB/1106556 for e5dc5c620be19b02666a19c280ff582e in 1002ms, sequenceid=264410324, compaction requested=true 2020-11-17 14:00:22,629 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 14:00:22,630 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/7e735b101c144c8d87efa654475d63e9, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5dbc2491b78345459cea98e9fd2163fe, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e110dc08d5a74ddda8eb097fbe93652d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=41.2 M 2020-11-17 14:00:22,658 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:00:22,661 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:00:22,685 INFO [Thread-4364] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:00:31,415 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#967 average throughput is 57.71 MB/second, slept 10 time(s) and total slept time is 2962 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 14:00:31,509 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into f8e18c4e184643b39534c11638993cc0(size=42.5 M), total size for store is 50.1 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 14:00:31,509 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605621622628; duration=8sec 2020-11-17 14:01:09,371 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 33264 ns. 2020-11-17 14:01:09,371 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14807 ns. 2020-11-17 14:01:42,288 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621606715 with entries=36672, filesize=128.03 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621702269 2020-11-17 14:02:02,520 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:02:07,013 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:02:39,188 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 61438 ns. 2020-11-17 14:02:53,531 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.74 MB heapSize=128.35 MB 2020-11-17 14:02:53,582 INFO [Thread-4372] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:02:54,203 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.74 MB at sequenceid=252625175 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/7c4c39fd163d431cb3c8f2fe22e56ef8 2020-11-17 14:02:54,209 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:02:54,226 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/7c4c39fd163d431cb3c8f2fe22e56ef8, entries=309070, sequenceid=252625175, filesize=6.4 M 2020-11-17 14:02:54,228 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.74 MB/111927915, heapSize ~128.35 MB/134582232, currentSize=21.03 KB/21534 for 82421ce8c5c8b48c573baee83caa8305 in 696ms, sequenceid=252625175, compaction requested=true 2020-11-17 14:03:00,749 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.61 MB heapSize=128.00 MB 2020-11-17 14:03:00,783 INFO [Thread-4374] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:03:01,536 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.61 MB at sequenceid=261139822 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/34fac4658a504b5a8a45d38ba6d163c7 2020-11-17 14:03:01,541 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:03:01,558 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/34fac4658a504b5a8a45d38ba6d163c7, entries=370166, sequenceid=261139822, filesize=7.1 M 2020-11-17 14:03:01,559 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~101.61 MB/106542166, heapSize ~128.00 MB/134218760, currentSize=74.56 KB/76350 for 06f7ca9534710e37d90d9f8345864259 in 810ms, sequenceid=261139822, compaction requested=true 2020-11-17 14:03:01,560 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 14:03:01,560 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/82759d654574476693ae34731187c6a5, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/6150ad1f6f2e481fa2cf84bc887f8b21, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/34fac4658a504b5a8a45d38ba6d163c7] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=50.2 M 2020-11-17 14:03:01,591 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:03:01,595 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:03:01,631 INFO [Thread-4376] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:03:08,540 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621702269 with entries=32189, filesize=128.26 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621788496 2020-11-17 14:03:14,379 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#970 average throughput is 55.26 MB/second, slept 13 time(s) and total slept time is 5219 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 14:03:14,408 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:03:14,433 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into bc62d8fa26f24c1eb6dae83cb75d964b(size=50.4 M), total size for store is 55.7 G. This selection was in queue for 0sec, and took 12sec to execute. 2020-11-17 14:03:14,433 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605621781559; duration=12sec 2020-11-17 14:03:18,094 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:03:22,333 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:03:39,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 45696 ns. 2020-11-17 14:03:42,034 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.68 GB, freeSize=158.33 MB, max=1.83 GB, blockCount=44348, accesses=42087891, hits=35623280, hitRatio=84.64%, , cachingAccesses=35676023, cachingHits=35616085, cachingHitsRatio=99.83%, evictions=5249, evicted=0, evictedPerRun=0.0 2020-11-17 14:04:40,305 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621788496 with entries=35004, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621880269 2020-11-17 14:05:09,652 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.00 MB heapSize=128.04 MB 2020-11-17 14:05:09,681 INFO [Thread-4382] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:05:10,449 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.00 MB at sequenceid=264425078 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/10d5df0b71494f0ab0db48f7994047d4 2020-11-17 14:05:10,455 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:05:10,473 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/10d5df0b71494f0ab0db48f7994047d4, entries=389885, sequenceid=264425078, filesize=8.0 M 2020-11-17 14:05:10,476 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.00 MB/104858172, heapSize ~128.04 MB/134261520, currentSize=237.17 KB/242864 for e5dc5c620be19b02666a19c280ff582e in 824ms, sequenceid=264425078, compaction requested=true 2020-11-17 14:05:15,187 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:05:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 206647 ns. 2020-11-17 14:05:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 396794 ns. 2020-11-17 14:05:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 59094 ns. 2020-11-17 14:06:07,760 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621880269 with entries=32582, filesize=128.42 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621967701 2020-11-17 14:06:11,945 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.89 MB heapSize=128.00 MB 2020-11-17 14:06:11,973 INFO [Thread-4386] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:12,717 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.89 MB at sequenceid=273202513 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/c19a94c2aa024c619857b97966293a23 2020-11-17 14:06:12,731 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:12,756 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/c19a94c2aa024c619857b97966293a23, entries=317727, sequenceid=273202513, filesize=8.7 M 2020-11-17 14:06:12,758 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~104.89 MB/109984380, heapSize ~128.00 MB/134218920, currentSize=67.19 KB/68799 for 0469ce862509756add4a8cdb00bc465c in 814ms, sequenceid=273202513, compaction requested=true 2020-11-17 14:06:12,759 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 14:06:12,759 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/5df77665faae42c2b51add5e51f44a59, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/e6d6d08014594692a937b585b34ee501, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/c19a94c2aa024c619857b97966293a23] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=62.5 M 2020-11-17 14:06:12,799 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:12,801 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:12,822 INFO [Thread-4388] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:18,764 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 29746 ns. 2020-11-17 14:06:18,764 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15604 ns. 2020-11-17 14:06:21,568 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.50 MB heapSize=128.08 MB 2020-11-17 14:06:21,597 INFO [Thread-4390] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:22,422 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.50 MB at sequenceid=273632557 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/94b1960233a348f19aa09c38df71c65c 2020-11-17 14:06:22,428 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:22,449 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/94b1960233a348f19aa09c38df71c65c, entries=282413, sequenceid=273632557, filesize=8.5 M 2020-11-17 14:06:22,450 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~107.50 MB/112722052, heapSize ~128.08 MB/134303384, currentSize=19.70 KB/20170 for fc93f6dcc564341dcc983bf30971f73d in 882ms, sequenceid=273632557, compaction requested=true 2020-11-17 14:06:22,451 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 14:06:22,451 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/7420b91c969040b5b09a6eed88318cef, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/f39b5cc8518e407d8e3aa08ca8029744, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/94b1960233a348f19aa09c38df71c65c] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=95.5 M 2020-11-17 14:06:22,484 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:22,487 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:22,508 INFO [Thread-4392] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:23,753 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:25,932 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:27,096 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:27,960 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#973 average throughput is 48.30 MB/second, slept 13 time(s) and total slept time is 7074 ms. 1 active operations remaining, total limit is 57.69 MB/second 2020-11-17 14:06:28,106 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into ba51705755824cf1a58af1df0105ed17(size=63.4 M), total size for store is 43.0 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 14:06:28,106 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605621972758; duration=15sec 2020-11-17 14:06:39,465 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:44,339 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:44,370 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:46,160 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:46,545 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:06:47,888 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#975 average throughput is 46.47 MB/second, slept 18 time(s) and total slept time is 10665 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 14:06:47,977 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into 76d6dab2e9e24011bd264a5ca2bd6753(size=95.6 M), total size for store is 33.6 G. This selection was in queue for 0sec, and took 25sec to execute. 2020-11-17 14:06:47,977 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605621982450; duration=25sec 2020-11-17 14:07:34,463 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.59 MB heapSize=128.13 MB 2020-11-17 14:07:34,496 INFO [Thread-4398] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:07:35,113 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.59 MB at sequenceid=252633686 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/746b246c097746a1877f9db840a64658 2020-11-17 14:07:35,122 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:07:35,144 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/746b246c097746a1877f9db840a64658, entries=308225, sequenceid=252633686, filesize=6.4 M 2020-11-17 14:07:35,146 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.59 MB/111763755, heapSize ~128.13 MB/134354536, currentSize=237.63 KB/243329 for 82421ce8c5c8b48c573baee83caa8305 in 683ms, sequenceid=252633686, compaction requested=true 2020-11-17 14:07:35,146 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 14:07:35,147 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/6c412cb04d694736a0a3e8fe9446bb62, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/7c4c39fd163d431cb3c8f2fe22e56ef8, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/746b246c097746a1877f9db840a64658] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=56.5 M 2020-11-17 14:07:35,168 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:07:35,197 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:07:35,223 INFO [Thread-4400] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:07:39,720 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621967701 with entries=34067, filesize=128.10 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622059686 2020-11-17 14:07:42,094 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:07:50,819 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#977 average throughput is 61.51 MB/second, slept 19 time(s) and total slept time is 7078 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 14:07:50,928 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 76da2fe0992b4c4c9601dc5b3a5ff6bc(size=56.5 M), total size for store is 11.6 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 14:07:50,929 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605622055146; duration=15sec 2020-11-17 14:07:59,116 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:08:23,375 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:08:28,417 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:08:32,188 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:08:38,751 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 38633 ns. 2020-11-17 14:08:49,508 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:08:49,627 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.89 MB heapSize=128.08 MB 2020-11-17 14:08:49,673 INFO [Thread-4406] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:08:50,661 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.89 MB at sequenceid=264436158 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/e4143281f3da4fef82e6d75f176def6e 2020-11-17 14:08:50,702 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e4143281f3da4fef82e6d75f176def6e, entries=405914, sequenceid=264436158, filesize=8.1 M 2020-11-17 14:08:50,704 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~98.89 MB/103690557, heapSize ~128.08 MB/134303824, currentSize=372.89 KB/381841 for e5dc5c620be19b02666a19c280ff582e in 1077ms, sequenceid=264436158, compaction requested=true 2020-11-17 14:08:50,706 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 14:08:50,706 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/f8e18c4e184643b39534c11638993cc0, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/10d5df0b71494f0ab0db48f7994047d4, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e4143281f3da4fef82e6d75f176def6e] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=58.6 M 2020-11-17 14:08:50,734 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:08:50,760 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:08:50,791 INFO [Thread-4408] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:08:52,339 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:08:58,434 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:09:01,011 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622059686 with entries=29656, filesize=128.69 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622140960 2020-11-17 14:09:03,936 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#979 average throughput is 53.45 MB/second, slept 13 time(s) and total slept time is 5271 ms. 0 active operations remaining, total limit is 53.85 MB/second 2020-11-17 14:09:03,990 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:09:04,012 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into e2eac97b94b941539d13f05cb2632542(size=60.0 M), total size for store is 50.1 G. This selection was in queue for 0sec, and took 13sec to execute. 2020-11-17 14:09:04,012 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605622130705; duration=13sec 2020-11-17 14:09:10,273 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:09:18,435 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:09:23,928 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.69 GB, freeSize=150.46 MB, max=1.83 GB, blockCount=44607, accesses=42756782, hits=36233991, hitRatio=84.74%, , cachingAccesses=36287013, cachingHits=36226768, cachingHitsRatio=99.83%, evictions=5279, evicted=0, evictedPerRun=0.0 2020-11-17 14:09:36,789 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:09:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 46749 ns. 2020-11-17 14:09:43,653 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:10:29,795 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622140960 with entries=33338, filesize=128.06 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622229750 2020-11-17 14:11:19,151 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 34918 ns. 2020-11-17 14:11:19,151 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 15226 ns. 2020-11-17 14:11:26,118 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.32 MB heapSize=128.01 MB 2020-11-17 14:11:26,172 INFO [Thread-4416] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:11:27,325 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.32 MB at sequenceid=263008197 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/57e92d7191dd40c296491558a86f1b2f 2020-11-17 14:11:27,331 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:11:27,350 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/57e92d7191dd40c296491558a86f1b2f, entries=585270, sequenceid=263008197, filesize=9.0 M 2020-11-17 14:11:27,351 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.32 MB/89464359, heapSize ~128.01 MB/134230816, currentSize=122.03 KB/124962 for 73aae5715542e1a5370f7540d2117abc in 1233ms, sequenceid=263008197, compaction requested=true 2020-11-17 14:11:27,352 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 14:11:27,352 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/a3dcbdf825d54d279f0eda98398d5efb, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/579008574a9e4a719941ec0c8ad61005, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/57e92d7191dd40c296491558a86f1b2f] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=63.8 M 2020-11-17 14:11:27,433 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:11:27,436 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:11:27,461 INFO [Thread-4418] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:11:35,543 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:11:37,166 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:11:38,028 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#981 average throughput is 55.16 MB/second, slept 10 time(s) and total slept time is 1770 ms. 0 active operations remaining, total limit is 53.85 MB/second 2020-11-17 14:11:38,129 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into efccc521c2c24444a85fb840360a390e(size=64.2 M), total size for store is 12.8 G. This selection was in queue for 0sec, and took 10sec to execute. 2020-11-17 14:11:38,129 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605622287351; duration=10sec 2020-11-17 14:11:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 410127 ns. 2020-11-17 14:11:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 197525 ns. 2020-11-17 14:11:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 53497 ns. 2020-11-17 14:11:50,318 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:11:56,366 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:12:00,501 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622229750 with entries=33351, filesize=128.12 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622320459 2020-11-17 14:12:08,535 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.53 MB heapSize=128.05 MB 2020-11-17 14:12:08,594 INFO [Thread-4423] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:12:09,486 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.53 MB at sequenceid=252641813 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/4ce01a2e2fa14eb9a1b3128ad514a39c 2020-11-17 14:12:09,494 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:12:09,512 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/4ce01a2e2fa14eb9a1b3128ad514a39c, entries=307814, sequenceid=252641813, filesize=6.4 M 2020-11-17 14:12:09,514 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.53 MB/111710039, heapSize ~128.05 MB/134273184, currentSize=628.63 KB/643720 for 82421ce8c5c8b48c573baee83caa8305 in 980ms, sequenceid=252641813, compaction requested=true 2020-11-17 14:12:15,899 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=89.46 MB heapSize=128.00 MB 2020-11-17 14:12:15,940 INFO [Thread-4425] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:12:17,106 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=89.46 MB at sequenceid=288897018 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/f1701cbe02d24383af32ec2de450150c 2020-11-17 14:12:17,114 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:12:17,140 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/f1701cbe02d24383af32ec2de450150c, entries=538946, sequenceid=288897018, filesize=9.0 M 2020-11-17 14:12:17,148 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~89.46 MB/93804992, heapSize ~128.00 MB/134218376, currentSize=283.27 KB/290071 for 5e5f05205d7e40b0141cd977dd009ba1 in 1249ms, sequenceid=288897018, compaction requested=true 2020-11-17 14:12:18,298 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=103.30 MB heapSize=128.29 MB 2020-11-17 14:12:18,332 INFO [Thread-4427] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:12:19,217 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=103.30 MB at sequenceid=264446900 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/5aa47958d976460c98706b25d3c7cb19 2020-11-17 14:12:19,263 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5aa47958d976460c98706b25d3c7cb19, entries=346015, sequenceid=264446900, filesize=8.1 M 2020-11-17 14:12:19,265 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~103.30 MB/108322817, heapSize ~128.29 MB/134517600, currentSize=1.24 MB/1302608 for e5dc5c620be19b02666a19c280ff582e in 967ms, sequenceid=264446900, compaction requested=true 2020-11-17 14:12:22,335 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:12:42,268 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:12:42,431 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:12:45,761 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:12:45,761 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:12:45,762 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:12:45,770 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:12:45,770 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:12:45,771 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:12:45,775 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:13:00,870 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:13:18,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:13:19,370 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.30 MB heapSize=128.15 MB 2020-11-17 14:13:19,400 INFO [Thread-4431] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:13:20,112 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.30 MB at sequenceid=261172198 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/b5508ce09c2f4bf7a139275790e960a3 2020-11-17 14:13:20,139 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/b5508ce09c2f4bf7a139275790e960a3, entries=369074, sequenceid=261172198, filesize=7.0 M 2020-11-17 14:13:20,140 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.30 MB/106224386, heapSize ~128.15 MB/134378960, currentSize=133.68 KB/136884 for 06f7ca9534710e37d90d9f8345864259 in 770ms, sequenceid=261172198, compaction requested=true 2020-11-17 14:13:25,042 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622320459 with entries=31687, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622405010 2020-11-17 14:13:26,848 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:13:32,828 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:13:41,983 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:14:01,354 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:14:12,127 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:14:18,616 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:14:24,164 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:14:36,031 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.69 GB, freeSize=150.12 MB, max=1.83 GB, blockCount=44618, accesses=43091863, hits=36559384, hitRatio=84.84%, , cachingAccesses=36612453, cachingHits=36552124, cachingHitsRatio=99.84%, evictions=5309, evicted=0, evictedPerRun=0.0 2020-11-17 14:14:38,825 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 41254 ns. 2020-11-17 14:14:38,826 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 74569 ns. 2020-11-17 14:14:55,372 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622405010 with entries=33490, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622495340 2020-11-17 14:15:12,350 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:15:43,413 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=87.70 MB heapSize=128.25 MB 2020-11-17 14:15:43,449 INFO [Thread-4439] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:15:44,576 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=87.70 MB at sequenceid=292724862 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/d33dde78cb85400ea42136a3fc8c5e0d 2020-11-17 14:15:44,584 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:15:44,602 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/d33dde78cb85400ea42136a3fc8c5e0d, entries=561734, sequenceid=292724862, filesize=8.6 M 2020-11-17 14:15:44,605 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~87.70 MB/91962891, heapSize ~128.25 MB/134475224, currentSize=50.29 KB/51499 for bdaa31a63b7f3d06d9909b7560333e79 in 1192ms, sequenceid=292724862, compaction requested=true 2020-11-17 14:15:44,606 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-17 14:15:44,607 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/daa59eeafbb74e71884bfee4f286b606, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/dad1b23bae454ef0ac01387a693f2143, hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/d33dde78cb85400ea42136a3fc8c5e0d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp, totalSize=78.6 M 2020-11-17 14:15:44,658 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:15:44,669 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:15:44,700 INFO [Thread-4441] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:15:50,473 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:15:51,509 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:15:51,521 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:15:53,999 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:15:54,001 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:15:54,071 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:15:55,395 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:15:55,401 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:15:55,403 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:15:57,696 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: bdaa31a63b7f3d06d9909b7560333e79#e#compaction#987 average throughput is 57.19 MB/second, slept 9 time(s) and total slept time is 2698 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 14:15:57,765 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of bdaa31a63b7f3d06d9909b7560333e79 into 712414c024334e50b32241273dac2572(size=77.1 M), total size for store is 31.9 G. This selection was in queue for 0sec, and took 13sec to execute. 2020-11-17 14:15:57,765 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., storeName=e, priority=11, startTime=1605622544605; duration=13sec 2020-11-17 14:16:10,714 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:16:20,101 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622495340 with entries=31704, filesize=128.09 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622580077 2020-11-17 14:16:20,102 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620876073 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605620876073 2020-11-17 14:16:20,118 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620931260 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605620931260 2020-11-17 14:16:20,124 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621006611 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605621006611 2020-11-17 14:16:20,135 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621058776 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605621058776 2020-11-17 14:16:20,140 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621114084 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605621114084 2020-11-17 14:16:20,145 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621187502 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605621187502 2020-11-17 14:16:20,155 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621247201 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605621247201 2020-11-17 14:16:20,160 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621425115 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605621425115 2020-11-17 14:16:20,165 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621524658 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605621524658 2020-11-17 14:16:20,170 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621606715 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605621606715 2020-11-17 14:16:20,175 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621702269 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605621702269 2020-11-17 14:16:20,180 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621788496 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605621788496 2020-11-17 14:16:20,185 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621880269 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605621880269 2020-11-17 14:16:28,866 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 32353 ns. 2020-11-17 14:16:28,866 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 12094 ns. 2020-11-17 14:16:48,037 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:16:59,193 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.90 MB heapSize=128.10 MB 2020-11-17 14:16:59,226 INFO [Thread-4448] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:16:59,925 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.90 MB at sequenceid=264461155 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/31116288c1a944f4a26ef98b0b966f2f 2020-11-17 14:16:59,931 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:16:59,950 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/31116288c1a944f4a26ef98b0b966f2f, entries=377081, sequenceid=264461155, filesize=8.1 M 2020-11-17 14:16:59,951 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.90 MB/105803835, heapSize ~128.10 MB/134320192, currentSize=128.36 KB/131445 for e5dc5c620be19b02666a19c280ff582e in 758ms, sequenceid=264461155, compaction requested=true 2020-11-17 14:16:59,952 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 14:16:59,952 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/e2eac97b94b941539d13f05cb2632542, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/5aa47958d976460c98706b25d3c7cb19, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/31116288c1a944f4a26ef98b0b966f2f] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=76.1 M 2020-11-17 14:16:59,986 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:16:59,988 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:17:00,017 INFO [Thread-4450] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:17:15,816 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#989 average throughput is 57.39 MB/second, slept 17 time(s) and total slept time is 5569 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 14:17:15,898 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 8d95111ea64146d5aba69b9fe52abf75(size=77.7 M), total size for store is 50.2 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 14:17:15,898 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605622619951; duration=15sec 2020-11-17 14:17:24,179 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.49 MB heapSize=128.25 MB 2020-11-17 14:17:24,203 INFO [Thread-4453] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:17:24,831 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.49 MB at sequenceid=252651239 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/4e33ddd59a274b989bec312d5cea2a81 2020-11-17 14:17:24,837 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:17:24,852 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/4e33ddd59a274b989bec312d5cea2a81, entries=310319, sequenceid=252651239, filesize=6.4 M 2020-11-17 14:17:24,853 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.49 MB/111657681, heapSize ~128.25 MB/134474992, currentSize=21.84 KB/22365 for 82421ce8c5c8b48c573baee83caa8305 in 674ms, sequenceid=252651239, compaction requested=true 2020-11-17 14:17:24,854 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 14:17:24,854 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/76da2fe0992b4c4c9601dc5b3a5ff6bc, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/4ce01a2e2fa14eb9a1b3128ad514a39c, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/4e33ddd59a274b989bec312d5cea2a81] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=69.3 M 2020-11-17 14:17:24,898 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:17:24,901 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:17:24,934 INFO [Thread-4455] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:17:38,741 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 56666 ns. 2020-11-17 14:17:38,742 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 211215 ns. 2020-11-17 14:17:38,742 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 413825 ns. 2020-11-17 14:17:45,799 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#991 average throughput is 56.18 MB/second, slept 22 time(s) and total slept time is 9036 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 14:17:45,876 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into d0bc8a112fb34472bb8797965d39e526(size=69.2 M), total size for store is 11.6 G. This selection was in queue for 0sec, and took 21sec to execute. 2020-11-17 14:17:45,877 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605622644854; duration=21sec 2020-11-17 14:17:56,980 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:18:00,827 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:18:04,195 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622580077 with entries=38568, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622684158 2020-11-17 14:18:15,544 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:18:56,597 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:19:26,365 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622684158 with entries=31200, filesize=128.24 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622766306 2020-11-17 14:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.69 GB, freeSize=151.41 MB, max=1.83 GB, blockCount=44591, accesses=43144035, hits=36565569, hitRatio=84.75%, , cachingAccesses=36618610, cachingHits=36558264, cachingHitsRatio=99.84%, evictions=5339, evicted=0, evictedPerRun=0.0 2020-11-17 14:19:39,327 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 61306 ns. 2020-11-17 14:19:39,327 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 41544 ns. 2020-11-17 14:19:43,581 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.70 MB heapSize=128.00 MB 2020-11-17 14:19:43,604 INFO [Thread-4464] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:19:44,350 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.70 MB at sequenceid=273242122 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/40061a7df6c941efa46b3f943deb2d73 2020-11-17 14:19:44,355 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:19:44,390 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/40061a7df6c941efa46b3f943deb2d73, entries=319624, sequenceid=273242122, filesize=8.7 M 2020-11-17 14:19:44,392 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.70 MB/109789705, heapSize ~128.00 MB/134217976, currentSize=351.86 KB/360306 for 0469ce862509756add4a8cdb00bc465c in 811ms, sequenceid=273242122, compaction requested=true 2020-11-17 14:19:56,012 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:20:39,374 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=98.46 MB heapSize=128.17 MB 2020-11-17 14:20:39,409 INFO [Thread-4468] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:20:40,318 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=98.46 MB at sequenceid=264472500 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/db0254c127ed4224bc85ea2a033ed4a5 2020-11-17 14:20:40,325 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:20:40,342 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/db0254c127ed4224bc85ea2a033ed4a5, entries=413511, sequenceid=264472500, filesize=8.1 M 2020-11-17 14:20:40,344 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~98.46 MB/103243900, heapSize ~128.17 MB/134397264, currentSize=361.35 KB/370019 for e5dc5c620be19b02666a19c280ff582e in 970ms, sequenceid=264472500, compaction requested=true 2020-11-17 14:20:46,187 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:21:03,450 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=106.94 MB heapSize=128.00 MB 2020-11-17 14:21:03,525 INFO [Thread-4471] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:21:04,154 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.94 MB at sequenceid=273659973 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/fec9dfe9b44840adb75e68766af16137 2020-11-17 14:21:04,162 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:21:04,180 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/fec9dfe9b44840adb75e68766af16137, entries=288232, sequenceid=273659973, filesize=8.4 M 2020-11-17 14:21:04,182 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.94 MB/112137979, heapSize ~128.00 MB/134217784, currentSize=18.65 KB/19099 for fc93f6dcc564341dcc983bf30971f73d in 732ms, sequenceid=273659973, compaction requested=true 2020-11-17 14:21:09,459 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622766306 with entries=37983, filesize=128.18 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622869434 2020-11-17 14:21:09,460 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621967701 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605621967701 2020-11-17 14:21:09,470 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622059686 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605622059686 2020-11-17 14:21:09,475 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622140960 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605622140960 2020-11-17 14:21:39,548 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 131445 ns. 2020-11-17 14:21:39,549 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14821 ns. 2020-11-17 14:22:32,256 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622869434 with entries=27586, filesize=128.13 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622952192 2020-11-17 14:22:39,129 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1330 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:22:39,129 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1259 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:22:39,130 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1244 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:22:39,133 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1227 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:22:39,133 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1215 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:22:39,133 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1214 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:22:39,134 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1196 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:22:39,134 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1176 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:22:39,134 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1165 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:22:39,134 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1134 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:22:39,135 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1041 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:22:39,135 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 998 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:22:39,135 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 971 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:22:39,160 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 983 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:22:39,160 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 952 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:22:42,970 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:23:03,572 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 625 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:03,572 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 614 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:03,578 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 617 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:03,690 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.53 MB heapSize=128.50 MB 2020-11-17 14:23:03,782 INFO [Thread-4478] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:23:04,924 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.55 MB at sequenceid=252660767 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/48eb101a939049f1b7a480adb84f1e53 2020-11-17 14:23:04,932 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:23:05,050 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/48eb101a939049f1b7a480adb84f1e53, entries=313889, sequenceid=252660767, filesize=6.5 M 2020-11-17 14:23:05,058 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.55 MB/111724811, heapSize ~128.53 MB/134774200, currentSize=492.28 KB/504091 for 82421ce8c5c8b48c573baee83caa8305 in 1367ms, sequenceid=252660767, compaction requested=true 2020-11-17 14:23:13,458 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 773 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:13,458 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 773 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:13,458 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 767 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:13,473 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 774 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:14,732 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:23:17,132 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1317 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:17,132 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1316 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:17,132 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1314 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:17,132 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1309 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:17,132 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1308 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:17,132 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1293 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:17,133 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1290 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:17,133 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1269 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:17,133 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1261 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:17,133 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1256 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:17,133 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1234 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:17,133 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1200 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:17,133 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1175 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:23:38,958 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 220923 ns. 2020-11-17 14:23:38,959 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 421906 ns. 2020-11-17 14:23:38,959 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 61089 ns. 2020-11-17 14:23:56,991 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:24:16,668 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622952192 with entries=28635, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623056623 2020-11-17 14:24:25,952 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:24:30,980 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=100.67 MB heapSize=128.01 MB 2020-11-17 14:24:31,138 INFO [Thread-4483] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:24:32,094 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.68 MB at sequenceid=261205269 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/56ba03ea06424327956400627be9edb6 2020-11-17 14:24:32,120 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:24:32,181 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/56ba03ea06424327956400627be9edb6, entries=378778, sequenceid=261205269, filesize=7.2 M 2020-11-17 14:24:32,192 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.68 MB/105571174, heapSize ~128.02 MB/134238768, currentSize=71.00 KB/72709 for 06f7ca9534710e37d90d9f8345864259 in 1213ms, sequenceid=261205269, compaction requested=true 2020-11-17 14:24:32,193 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 14:24:32,193 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/bc62d8fa26f24c1eb6dae83cb75d964b, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/b5508ce09c2f4bf7a139275790e960a3, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/56ba03ea06424327956400627be9edb6] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=64.6 M 2020-11-17 14:24:32,285 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:24:32,290 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:24:32,328 INFO [Thread-4485] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:24:32,635 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.69 GB, freeSize=151.53 MB, max=1.83 GB, blockCount=44589, accesses=43155908, hits=36573827, hitRatio=84.75%, , cachingAccesses=36626866, cachingHits=36566512, cachingHitsRatio=99.84%, evictions=5369, evicted=0, evictedPerRun=0.0 2020-11-17 14:24:36,925 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.98 MB heapSize=128.17 MB 2020-11-17 14:24:36,989 INFO [Thread-4487] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:24:38,506 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.98 MB at sequenceid=264482726 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/6a6e0be879f34be29e00a1e6da9b40f4 2020-11-17 14:24:38,512 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:24:38,537 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6a6e0be879f34be29e00a1e6da9b40f4, entries=377178, sequenceid=264482726, filesize=8.1 M 2020-11-17 14:24:38,543 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.98 MB/105889476, heapSize ~128.17 MB/134391888, currentSize=373.61 KB/382577 for e5dc5c620be19b02666a19c280ff582e in 1618ms, sequenceid=264482726, compaction requested=true 2020-11-17 14:24:38,544 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 14:24:38,544 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8d95111ea64146d5aba69b9fe52abf75, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/db0254c127ed4224bc85ea2a033ed4a5, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/6a6e0be879f34be29e00a1e6da9b40f4] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=93.9 M 2020-11-17 14:24:38,584 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:24:38,586 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:24:38,612 INFO [Thread-4489] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:24:39,902 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:24:48,589 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 837 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:24:48,589 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 836 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:24:49,819 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 795 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:24:49,819 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 786 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:24:49,819 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 781 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:24:49,819 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 780 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:24:49,819 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 772 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:24:49,820 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 769 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:24:49,820 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 768 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:24:51,867 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1136 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:24:51,868 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1132 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:24:51,868 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1132 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:24:57,825 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#997 average throughput is 35.54 MB/second, slept 15 time(s) and total slept time is 10823 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 14:24:57,891 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:24:57,919 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 29ac6b53e458492691bd7afd6f401320(size=64.9 M), total size for store is 55.7 G. This selection was in queue for 0sec, and took 25sec to execute. 2020-11-17 14:24:57,919 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605623072192; duration=25sec 2020-11-17 14:25:02,322 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 500 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:03,944 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 669 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:03,944 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 668 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:03,944 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 666 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:04,141 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:25:05,643 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 666 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:05,643 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 664 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:05,643 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 663 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:05,643 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 661 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:07,277 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 672 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:07,278 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 669 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:07,278 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 669 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:08,855 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 610 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:08,856 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 609 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:08,856 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 605 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:08,856 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 604 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:10,276 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 529 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:10,620 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#999 average throughput is 34.58 MB/second, slept 14 time(s) and total slept time is 9003 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 14:25:10,744 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 601a862513114cb388e20f19ce68f60b(size=95.3 M), total size for store is 50.2 G. This selection was in queue for 0sec, and took 32sec to execute. 2020-11-17 14:25:10,744 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605623078543; duration=32sec 2020-11-17 14:25:22,743 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 973 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:22,743 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 973 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:22,743 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 972 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:22,743 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 972 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:22,744 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 972 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:22,744 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 972 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:22,744 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 972 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:22,744 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 945 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:22,744 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 930 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:22,744 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 914 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:22,744 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 904 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:22,744 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 901 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:22,744 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 901 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:29,024 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:25:29,212 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623056623 with entries=21481, filesize=128.66 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623129094 2020-11-17 14:25:31,845 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 582 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:31,846 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 580 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 72969 ns. 2020-11-17 14:25:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 41287 ns. 2020-11-17 14:25:45,660 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 586 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:45,660 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 586 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:25:45,660 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 586 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:26:21,626 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:26:33,637 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 817 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:26:33,637 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 811 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:26:33,644 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 817 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:26:41,042 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.99 MB heapSize=128.07 MB 2020-11-17 14:26:41,289 INFO [Thread-4497] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:26:43,114 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.99 MB at sequenceid=263052788 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/9e3acc949d64427d9abafb1868cf7042 2020-11-17 14:26:43,184 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:26:43,338 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/9e3acc949d64427d9abafb1868cf7042, entries=572701, sequenceid=263052788, filesize=8.8 M 2020-11-17 14:26:43,380 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~85.99 MB/90169145, heapSize ~128.07 MB/134294704, currentSize=116.87 KB/119679 for 73aae5715542e1a5370f7540d2117abc in 2339ms, sequenceid=263052788, compaction requested=true 2020-11-17 14:26:45,558 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 647 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:26:46,593 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:26:48,998 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 23587 ns. 2020-11-17 14:26:48,998 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 33556 ns. 2020-11-17 14:27:00,893 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623129094 with entries=28534, filesize=128.18 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623220852 2020-11-17 14:27:00,893 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622229750 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605622229750 2020-11-17 14:27:14,343 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:27:24,589 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:27:45,970 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 792 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:27:45,970 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 789 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:27:45,970 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 779 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:27:45,970 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 773 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:27:45,970 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 772 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:27:49,186 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.31 MB heapSize=128.00 MB 2020-11-17 14:27:49,313 INFO [Thread-4502] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:27:50,709 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1247 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:27:50,709 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1243 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:27:50,709 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1241 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:27:52,161 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.31 MB at sequenceid=252667006 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/a01a01a819dc4d04bd13043e5b0469e5 2020-11-17 14:27:52,178 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:27:52,274 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a01a01a819dc4d04bd13043e5b0469e5, entries=309999, sequenceid=252667006, filesize=6.4 M 2020-11-17 14:27:52,283 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.31 MB/111477148, heapSize ~128.00 MB/134219432, currentSize=2.08 MB/2178440 for 82421ce8c5c8b48c573baee83caa8305 in 3097ms, sequenceid=252667006, compaction requested=true 2020-11-17 14:27:52,285 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 14:27:52,286 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d0bc8a112fb34472bb8797965d39e526, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/48eb101a939049f1b7a480adb84f1e53, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/a01a01a819dc4d04bd13043e5b0469e5] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=82.1 M 2020-11-17 14:27:53,081 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:27:53,088 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:27:53,145 INFO [Thread-4504] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:27:57,577 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:28:01,716 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 581 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:01,716 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 581 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:01,722 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 582 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:01,723 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 582 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:01,723 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 580 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:01,723 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 580 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:01,723 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 580 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:02,239 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.21 MB heapSize=128.00 MB 2020-11-17 14:28:02,980 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 706 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:03,053 INFO [Thread-4506] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:28:04,635 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 818 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:05,989 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 709 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:05,991 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 703 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:05,991 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 702 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:06,205 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.21 MB at sequenceid=264490932 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/d3498e4600224d6cb351d4a0efc7052d 2020-11-17 14:28:06,250 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:28:06,369 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d3498e4600224d6cb351d4a0efc7052d, entries=357415, sequenceid=264490932, filesize=8.0 M 2020-11-17 14:28:06,388 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.21 MB/107177746, heapSize ~128.00 MB/134218040, currentSize=1.66 MB/1738033 for e5dc5c620be19b02666a19c280ff582e in 4150ms, sequenceid=264490932, compaction requested=true 2020-11-17 14:28:07,288 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 630 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:07,288 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 630 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:07,300 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 639 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:07,300 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 633 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:07,303 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 621 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:08,154 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623220852 with entries=17406, filesize=129.11 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623287962 2020-11-17 14:28:12,946 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 721 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:12,947 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 709 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:12,947 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 707 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:12,947 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 707 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:12,947 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 704 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:12,947 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 702 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:18,759 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:28:18,769 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:28:19,323 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:28:19,328 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:28:20,474 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#1002 average throughput is 50.54 MB/second, slept 16 time(s) and total slept time is 5061 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 14:28:20,570 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 93173a529bfe4e4b9eb48e7262d35220(size=81.8 M), total size for store is 11.6 G. This selection was in queue for 0sec, and took 28sec to execute. 2020-11-17 14:28:20,570 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605623272284; duration=28sec 2020-11-17 14:28:35,622 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 878 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:35,622 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 873 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:35,622 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 859 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:35,622 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 853 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:28:35,623 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 851 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:29:15,274 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623287962 with entries=21212, filesize=129.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623354918 2020-11-17 14:29:17,234 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1380 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:29:17,234 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1380 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:29:17,234 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1368 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:29:17,234 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1354 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:29:17,234 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1321 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:29:17,235 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1308 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:29:17,235 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1227 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:29:17,235 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1225 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:29:17,235 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1214 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:29:17,235 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1166 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:29:17,235 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1105 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:29:17,235 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1097 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:29:17,235 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1091 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:29:17,235 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1073 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:29:24,922 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.69 GB, freeSize=151.02 MB, max=1.83 GB, blockCount=44592, accesses=43345203, hits=36711553, hitRatio=84.70%, , cachingAccesses=36764621, cachingHits=36704229, cachingHitsRatio=99.84%, evictions=5399, evicted=0, evictedPerRun=0.0 2020-11-17 14:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 14:29:38,850 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 222621 ns. 2020-11-17 14:29:38,851 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 470638 ns. 2020-11-17 14:29:38,851 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 54699 ns. 2020-11-17 14:29:51,754 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=107.24 MB heapSize=128.92 MB 2020-11-17 14:29:51,806 INFO [Thread-4511] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:29:52,444 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.24 MB at sequenceid=252669843 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/b3f1d1d8da5e4382ab1568732ef332a6 2020-11-17 14:29:52,453 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:29:52,476 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b3f1d1d8da5e4382ab1568732ef332a6, entries=310383, sequenceid=252669843, filesize=6.2 M 2020-11-17 14:29:52,478 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~107.24 MB/112450888, heapSize ~128.92 MB/135181400, currentSize=1.17 MB/1223142 for 82421ce8c5c8b48c573baee83caa8305 in 724ms, sequenceid=252669843, compaction requested=true 2020-11-17 14:30:16,887 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1120 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:30:16,890 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1111 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:30:16,890 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1109 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:30:16,890 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1108 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:30:16,890 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1106 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:30:16,890 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1101 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:30:16,890 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1098 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:30:16,890 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1067 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:30:16,890 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1066 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:30:16,890 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1065 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:30:16,890 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1064 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:30:16,890 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1059 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:30:16,890 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1006 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:30:16,890 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 977 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:30:16,890 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 950 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:30:20,309 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:30:21,118 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623354918 with entries=20338, filesize=128.95 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623421078 2020-11-17 14:30:38,727 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 83504 ns. 2020-11-17 14:30:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 62607 ns. 2020-11-17 14:31:04,123 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 995 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:31:04,123 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 995 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:31:04,123 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 994 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:31:15,591 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1000 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:31:15,592 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1000 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:31:15,592 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 997 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:31:15,592 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 995 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:31:15,592 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 995 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:31:15,592 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 992 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:31:15,592 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 989 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:31:16,887 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623421078 with entries=15359, filesize=128.94 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623476732 2020-11-17 14:31:17,357 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:31:29,696 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=102.16 MB heapSize=128.00 MB 2020-11-17 14:31:29,959 INFO [Thread-4517] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:31:30,997 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.16 MB at sequenceid=261222871 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/7864c3ce89364e559c3bfe83b25fdbf0 2020-11-17 14:31:31,025 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:31:31,134 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/7864c3ce89364e559c3bfe83b25fdbf0, entries=361827, sequenceid=261222871, filesize=6.7 M 2020-11-17 14:31:31,153 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.16 MB/107124031, heapSize ~128.00 MB/134218496, currentSize=147.08 KB/150607 for 06f7ca9534710e37d90d9f8345864259 in 1456ms, sequenceid=261222871, compaction requested=true 2020-11-17 14:31:32,510 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.51 MB heapSize=128.01 MB 2020-11-17 14:31:32,680 INFO [Thread-4519] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:31:34,905 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1103 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:31:34,916 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1112 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:31:34,969 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1164 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:31:35,340 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.51 MB at sequenceid=264499723 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/c9fb2444d13b40a299ad38be4d22d37b 2020-11-17 14:31:35,387 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:31:35,481 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c9fb2444d13b40a299ad38be4d22d37b, entries=381021, sequenceid=264499723, filesize=8.0 M 2020-11-17 14:31:35,496 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~100.51 MB/105389871, heapSize ~128.01 MB/134226704, currentSize=107.78 KB/110371 for e5dc5c620be19b02666a19c280ff582e in 2987ms, sequenceid=264499723, compaction requested=true 2020-11-17 14:31:35,497 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 14:31:35,498 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/601a862513114cb388e20f19ce68f60b, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/d3498e4600224d6cb351d4a0efc7052d, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c9fb2444d13b40a299ad38be4d22d37b] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=111.3 M 2020-11-17 14:31:35,743 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:31:35,760 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:31:35,900 INFO [Thread-4521] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:31:38,295 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 928 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:31:42,407 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 658 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:31:44,252 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 741 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:31:44,252 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 734 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:31:58,930 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 20649 ns. 2020-11-17 14:31:58,930 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 58170 ns. 2020-11-17 14:32:00,400 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:32:00,405 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:32:01,902 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#1007 average throughput is 50.15 MB/second, slept 20 time(s) and total slept time is 6088 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 14:32:02,350 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into 6bc286edb4864e10823af48a0308b91a(size=112.5 M), total size for store is 50.2 G. This selection was in queue for 0sec, and took 26sec to execute. 2020-11-17 14:32:02,350 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=11, startTime=1605623495496; duration=26sec 2020-11-17 14:32:03,398 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 727 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:21,555 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.63 MB heapSize=128.08 MB 2020-11-17 14:32:21,598 INFO [Thread-4526] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:32:22,186 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.63 MB at sequenceid=252673078 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/1359f24883bd4e27a2e852291803f263 2020-11-17 14:32:22,192 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:32:22,209 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/1359f24883bd4e27a2e852291803f263, entries=306833, sequenceid=252673078, filesize=6.2 M 2020-11-17 14:32:22,210 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.63 MB/111811375, heapSize ~128.08 MB/134299688, currentSize=761.80 KB/780083 for 82421ce8c5c8b48c573baee83caa8305 in 655ms, sequenceid=252673078, compaction requested=true 2020-11-17 14:32:22,211 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 14:32:22,211 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/93173a529bfe4e4b9eb48e7262d35220, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/b3f1d1d8da5e4382ab1568732ef332a6, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/1359f24883bd4e27a2e852291803f263] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=94.2 M 2020-11-17 14:32:22,237 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:32:22,240 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:32:22,265 INFO [Thread-4528] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:32:22,299 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623476732 with entries=19579, filesize=128.05 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623542268 2020-11-17 14:32:30,951 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 583 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:30,951 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 581 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:34,300 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1786 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:34,300 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1783 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:34,300 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1778 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:34,300 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1761 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:34,300 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1761 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:34,300 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1759 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:34,300 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1758 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:34,300 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1754 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:35,799 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 719 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:35,800 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 719 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:35,808 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 723 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:35,808 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 715 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:35,808 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 688 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:35,808 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 684 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:35,808 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 674 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:37,127 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 677 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:37,128 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 676 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:37,128 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 674 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:37,128 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 675 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:37,128 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 659 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:40,625 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 564 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:40,625 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 563 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:40,629 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 562 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:40,629 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 560 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:42,348 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 851 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:42,349 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 847 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:42,349 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 847 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:42,349 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 846 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:42,351 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 845 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:42,351 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 845 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:44,092 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 777 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:44,103 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 784 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:32:51,309 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:32:51,385 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:32:51,388 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:32:52,791 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:32:53,257 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#1009 average throughput is 51.37 MB/second, slept 22 time(s) and total slept time is 7928 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 14:32:53,302 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:32:53,322 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 66455d2901fc4fdd872fc772b00171ef(size=93.8 M), total size for store is 11.6 G. This selection was in queue for 0sec, and took 31sec to execute. 2020-11-17 14:32:53,323 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605623542210; duration=31sec 2020-11-17 14:32:57,638 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:33:05,792 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=105.11 MB heapSize=128.27 MB 2020-11-17 14:33:05,836 INFO [Thread-4532] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:33:06,597 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.11 MB at sequenceid=273275254 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/18b9f8645ada455ba73ff819256ea50d 2020-11-17 14:33:06,603 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:33:06,618 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/18b9f8645ada455ba73ff819256ea50d, entries=318417, sequenceid=273275254, filesize=8.6 M 2020-11-17 14:33:06,620 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.11 MB/110213513, heapSize ~128.27 MB/134503032, currentSize=17.79 KB/18212 for 0469ce862509756add4a8cdb00bc465c in 828ms, sequenceid=273275254, compaction requested=true 2020-11-17 14:33:06,621 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 14:33:06,621 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/ba51705755824cf1a58af1df0105ed17, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/40061a7df6c941efa46b3f943deb2d73, hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/18b9f8645ada455ba73ff819256ea50d] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp, totalSize=80.7 M 2020-11-17 14:33:06,656 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:33:06,659 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:33:06,679 INFO [Thread-4534] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:33:13,642 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:33:14,535 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623542268 with entries=17446, filesize=128.24 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623594499 2020-11-17 14:33:18,089 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:33:20,090 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:33:21,681 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 0469ce862509756add4a8cdb00bc465c#e#compaction#1011 average throughput is 62.43 MB/second, slept 16 time(s) and total slept time is 6259 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 14:33:21,740 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 0469ce862509756add4a8cdb00bc465c into 25cded4203954a6ea94220af3bca03c2(size=81.3 M), total size for store is 43.1 G. This selection was in queue for 0sec, and took 15sec to execute. 2020-11-17 14:33:21,740 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., storeName=e, priority=10, startTime=1605623586620; duration=15sec 2020-11-17 14:33:25,098 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:33:25,103 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:33:25,104 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:33:36,598 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:33:56,041 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623594499 with entries=13525, filesize=128.61 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623635919 2020-11-17 14:34:03,983 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.91 MB heapSize=128.26 MB 2020-11-17 14:34:04,032 INFO [Thread-4540] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:34:05,456 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 838 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:34:05,456 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 836 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:34:05,456 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 834 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:34:05,617 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.91 MB at sequenceid=252675793 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/e73ce9f38db74c5788476ca8b885924c 2020-11-17 14:34:05,650 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:34:05,758 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/e73ce9f38db74c5788476ca8b885924c, entries=305575, sequenceid=252675793, filesize=6.2 M 2020-11-17 14:34:05,765 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.91 MB/112104967, heapSize ~128.26 MB/134492200, currentSize=398.71 KB/408278 for 82421ce8c5c8b48c573baee83caa8305 in 1782ms, sequenceid=252675793, compaction requested=true 2020-11-17 14:34:16,937 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=89.33 MB heapSize=128.00 MB 2020-11-17 14:34:16,982 INFO [Thread-4543] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:34:17,987 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=89.33 MB at sequenceid=288961414 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/414df425f8904367b294cc4d9b74f7ed 2020-11-17 14:34:17,993 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:34:18,011 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/414df425f8904367b294cc4d9b74f7ed, entries=540020, sequenceid=288961414, filesize=8.9 M 2020-11-17 14:34:18,012 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~89.33 MB/93671826, heapSize ~128.00 MB/134217952, currentSize=40.37 KB/41337 for 5e5f05205d7e40b0141cd977dd009ba1 in 1075ms, sequenceid=288961414, compaction requested=true 2020-11-17 14:34:18,013 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 14:34:18,013 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/f7a2afe80a8746008d7160e8a7b397ad, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/f1701cbe02d24383af32ec2de450150c, hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/414df425f8904367b294cc4d9b74f7ed] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp, totalSize=63.6 M 2020-11-17 14:34:18,041 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:34:18,046 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:34:18,071 INFO [Thread-4545] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:34:26,210 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:34:26,316 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:34:27,876 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:34:27,885 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:34:28,656 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 5e5f05205d7e40b0141cd977dd009ba1#e#compaction#1014 average throughput is 57.59 MB/second, slept 11 time(s) and total slept time is 2415 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 14:34:28,715 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 5e5f05205d7e40b0141cd977dd009ba1 into 2c719636c7334beebd560893ad815839(size=63.6 M), total size for store is 24.5 G. This selection was in queue for 0sec, and took 10sec to execute. 2020-11-17 14:34:28,715 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., storeName=e, priority=10, startTime=1605623658013; duration=10sec 2020-11-17 14:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=116.55 MB, max=1.83 GB, blockCount=45802, accesses=43546317, hits=36839212, hitRatio=84.60%, , cachingAccesses=36893478, cachingHits=36831828, cachingHitsRatio=99.83%, evictions=5429, evicted=0, evictedPerRun=0.0 2020-11-17 14:35:10,950 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 2072 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:35:20,927 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 9614 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:35:21,378 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 10381 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:35:21,379 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 10371 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:35:21,379 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 10364 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:35:21,379 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 10123 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:35:21,588 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623710884","responsesize":"24","method":"Multi","param":"region\u003d aeris_v2,\\x06\\xCD\\x04\\x16\\x97\\xC4\\x05Smash\\xC4\\x07routing\\xC4\\x03pim\\xC4\\x0Asparsemode\\xC4\\x06status\\xC4\\x07default\\xC4\\x09routeIntf,160496158 \u003cTRUNCATED\u003e","processingtimems":10500,"client":"192.168.2.22:47084","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 14:35:21,588 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623711006","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x9A\\xC4\\x07Devices\\xC4\\x0BJPE15164024\\ \u003cTRUNCATED\u003e","processingtimems":10379,"client":"192.168.2.21:57988","queuetimems":93,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:35:21,591 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623710885","responsesize":"38","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 5 action(s) and 1st row key\u003d\\x0B\\x0E\\x97\\xC4\\x05Sysdb\\xC4\\x0Benvironment\\xC \u003cTRUNCATED\u003e","processingtimems":10508,"client":"192.168.2.22:47084","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":5} 2020-11-17 14:35:21,607 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623710884","responsesize":"222","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 34 action(s) and 1st row key\u003d\\x04\\xCD\\x04 \\x97\\xC4\\x05Sysdb\\xC4\\x0Benvironm \u003cTRUNCATED\u003e","processingtimems":10575,"client":"192.168.2.22:47084","queuetimems":2,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":34} 2020-11-17 14:35:21,588 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623708857","responsesize":"1570","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 239 action(s) and 1st row key\u003d\\x03\\xCD\\x06\\x11\\x94\\xC4\\x06Kernel\\xC4\\x04pro \u003cTRUNCATED\u003e","processingtimems":12569,"client":"192.168.2.23:38072","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":239} 2020-11-17 14:35:21,618 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623711000","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":10382,"client":"192.168.2.22:47114","queuetimems":108,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:35:21,618 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623710884","responsesize":"172","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 26 action(s) and 1st row key\u003d\\x02\\xCD\\x04!\\x9A\\xC4\\x05Sysdb\\xC4\\x0Bl2discov \u003cTRUNCATED\u003e","processingtimems":10557,"client":"192.168.2.22:47084","queuetimems":3,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":26} 2020-11-17 14:35:21,608 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623710884","responsesize":"503","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 81 action(s) and 1st row key\u003d\\x04\\xCD\\x04\\x1D\\x96\\xC4\\x05Smash\\xC4\\x07routi \u003cTRUNCATED\u003e","processingtimems":10576,"client":"192.168.2.22:47084","queuetimems":2,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":81} 2020-11-17 14:35:21,649 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623710885","responsesize":"32","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 4 action(s) and 1st row key\u003d\\x09\\xCD\\x06 \\x98\\xC4\\x05Sysdb\\xC4\\x0Benvironme \u003cTRUNCATED\u003e","processingtimems":10507,"client":"192.168.2.22:47084","queuetimems":3,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":4} 2020-11-17 14:35:21,640 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623711000","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":10385,"client":"192.168.2.22:47114","queuetimems":108,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:35:21,639 INFO [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1885ms GC pool 'G1 Young Generation' had collection(s): count=1 time=1848ms 2020-11-17 14:35:21,638 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623710885","responsesize":"18","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 2 action(s) and 1st row key\u003d\\x07$\\x97\\xC4\\x05Sysdb\\xC4\\x02ip\\xC4\\x06helper\\ \u003cTRUNCATED\u003e","processingtimems":10503,"client":"192.168.2.22:47084","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 14:35:21,637 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623711006","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH14305100\\ \u003cTRUNCATED\u003e","processingtimems":10377,"client":"192.168.2.21:57988","queuetimems":93,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:35:21,636 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623710996","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":10388,"client":"192.168.2.22:47114","queuetimems":104,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:35:21,695 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623710884","responsesize":"24","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 3 action(s) and 1st row key\u003d\\x0B\\xCD\\x04\\x03\\x96\\xC4\\x05Sysdb\\xC4\\x0Benviro \u003cTRUNCATED\u003e","processingtimems":10504,"client":"192.168.2.22:47084","queuetimems":2,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 14:35:21,695 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623711013","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x07Devices\\xC4\\x0BJPE14192364\\ \u003cTRUNCATED\u003e","processingtimems":10372,"client":"192.168.2.21:57988","queuetimems":100,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:35:21,694 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623708845","responsesize":"240","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 37 action(s) and 1st row key\u003d\\x0B\\xCD\\x06\\x15\\x98\\xC4\\x05Sysdb\\xC4\\x0Benvir \u003cTRUNCATED\u003e","processingtimems":12552,"client":"192.168.2.22:47084","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":37} 2020-11-17 14:35:36,543 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623635919 with entries=32922, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623736505 2020-11-17 14:35:36,545 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622320459 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605622320459 2020-11-17 14:35:36,558 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622405010 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605622405010 2020-11-17 14:35:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 77556 ns. 2020-11-17 14:35:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 257784 ns. 2020-11-17 14:35:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 463172 ns. 2020-11-17 14:35:43,993 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.24 MB heapSize=128.00 MB 2020-11-17 14:35:44,027 INFO [Thread-4551] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:35:44,793 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.24 MB at sequenceid=273680400 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/ab5778c3c9a74ce2a75ab652e6b986e3 2020-11-17 14:35:44,802 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:35:44,826 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/ab5778c3c9a74ce2a75ab652e6b986e3, entries=285245, sequenceid=273680400, filesize=8.4 M 2020-11-17 14:35:44,828 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~107.24 MB/112454490, heapSize ~128.00 MB/134220400, currentSize=709.36 KB/726385 for fc93f6dcc564341dcc983bf30971f73d in 835ms, sequenceid=273680400, compaction requested=true 2020-11-17 14:35:44,829 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 14:35:44,829 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/76d6dab2e9e24011bd264a5ca2bd6753, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/fec9dfe9b44840adb75e68766af16137, hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/ab5778c3c9a74ce2a75ab652e6b986e3] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp, totalSize=112.4 M 2020-11-17 14:35:44,872 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:35:44,878 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:35:44,908 INFO [Thread-4553] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:35:49,061 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=97.80 MB heapSize=128.12 MB 2020-11-17 14:35:49,100 INFO [Thread-4555] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:35:50,157 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.80 MB at sequenceid=264511009 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/8247264ead944fdc8921cddc4e2aab2d 2020-11-17 14:35:50,165 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:35:50,183 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8247264ead944fdc8921cddc4e2aab2d, entries=421340, sequenceid=264511009, filesize=7.9 M 2020-11-17 14:35:50,185 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~97.80 MB/102546808, heapSize ~128.12 MB/134340984, currentSize=213.81 KB/218941 for e5dc5c620be19b02666a19c280ff582e in 1124ms, sequenceid=264511009, compaction requested=true 2020-11-17 14:36:02,740 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:36:02,742 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:36:04,224 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:36:06,710 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:36:08,752 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: fc93f6dcc564341dcc983bf30971f73d#e#compaction#1016 average throughput is 58.18 MB/second, slept 27 time(s) and total slept time is 11088 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 14:36:09,046 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of fc93f6dcc564341dcc983bf30971f73d into f1d6dfa258d145bea7a1e197a8dc46ff(size=112.4 M), total size for store is 33.6 G. This selection was in queue for 0sec, and took 24sec to execute. 2020-11-17 14:36:09,047 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d., storeName=e, priority=10, startTime=1605623744828; duration=24sec 2020-11-17 14:36:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 81129 ns. 2020-11-17 14:36:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 43624 ns. 2020-11-17 14:36:47,976 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623736505 with entries=26504, filesize=128.41 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623807936 2020-11-17 14:36:48,029 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:37:08,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14679 ns. 2020-11-17 14:37:08,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 23853 ns. 2020-11-17 14:37:25,550 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:37:31,841 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 899 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:37:31,841 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 893 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:37:31,841 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 883 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:37:31,842 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 875 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:37:31,842 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 873 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:37:31,842 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 862 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:37:31,842 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 853 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:37:31,842 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 847 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:37:31,842 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 824 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:37:31,842 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 815 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:37:31,842 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 803 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:37:31,842 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 777 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:37:31,842 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 769 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:37:33,922 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 877 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:37:53,902 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Connection to 192.168.2.23/192.168.2.23:15010 closed at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$channelInactive$2(FanOutOneBlockAsyncDFSOutput.java:289) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.channelInactive(FanOutOneBlockAsyncDFSOutput.java:288) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:242) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:228) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:221) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:242) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:228) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:221) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:390) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:355) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:242) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:228) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:221) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelInactive(IdleStateHandler.java:277) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:242) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:228) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:221) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1403) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:242) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:228) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:912) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:827) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:37:53,928 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Connection to 192.168.2.23/192.168.2.23:15010 closed at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$channelInactive$2(FanOutOneBlockAsyncDFSOutput.java:289) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.channelInactive(FanOutOneBlockAsyncDFSOutput.java:288) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:242) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:228) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:221) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:242) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:228) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:221) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:390) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:355) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:242) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:228) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:221) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelInactive(IdleStateHandler.java:277) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:242) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:228) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:221) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1403) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:242) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:228) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:912) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:827) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:37:55,056 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623807936 with entries=18623, filesize=85.37 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623873929 2020-11-17 14:37:55,130 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623862574","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":12515,"client":"192.168.2.22:47114","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:37:55,132 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623862595","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4! \u003cTRUNCATED\u003e","processingtimems":12489,"client":"192.168.2.23:38618","queuetimems":17,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:37:55,134 WARN [Close-WAL-Writer-533] wal.AbstractProtobufLogWriter: Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:424) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:513) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:220) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(AsyncProtobufLogWriter.java:176) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:211) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:235) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:153) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$6(AsyncFSWAL.java:672) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:37:55,152 WARN [Close-WAL-Writer-533] wal.AsyncProtobufLogWriter: normal close failed, try recover java.lang.IllegalStateException: should call flush first before calling close at org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkState(Preconditions.java:508) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:518) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:569) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:154) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$6(AsyncFSWAL.java:672) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:37:55,230 INFO [Close-WAL-Writer-533] util.FSHDFSUtils: Recover lease on dfs file /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623807936 2020-11-17 14:37:55,252 INFO [Close-WAL-Writer-533] util.FSHDFSUtils: Failed to recover lease, attempt=0 on file=/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623807936 after 22ms 2020-11-17 14:37:55,360 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623862184","responsesize":"3448","method":"Multi","param":"region\u003d aeris_v2,\\x11\\xCD\\x04\\x0C\\x96\\xC4\\x05Smash\\xC4\\x08routing6\\xC4\\x03pim\\xC4\\x06status\\xC4\\x07default\\xC4\\x14intfSecondaryAddress,1604670627883.8 \u003cTRUNCATED\u003e","processingtimems":13166,"client":"192.168.2.23:38072","queuetimems":2,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":507} 2020-11-17 14:37:55,436 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623862450","responsesize":"420","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 67 action(s) and 1st row key\u003d\\x02\\xCD\\x06\\x1B\\x94\\xC4\\x06Kernel\\xC4\\x04proc \u003cTRUNCATED\u003e","processingtimems":12874,"client":"192.168.2.23:38072","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":67} 2020-11-17 14:37:57,836 INFO [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1156ms GC pool 'G1 Young Generation' had collection(s): count=1 time=1479ms 2020-11-17 14:37:57,870 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1588 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:37:59,259 INFO [Close-WAL-Writer-533] util.FSHDFSUtils: Recovered lease, attempt=1 on file=/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623807936 after 4029ms 2020-11-17 14:38:05,782 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 5224 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:05,811 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 5277 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:24,243 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:24,381 INFO [regionserver/cvp21:16201.Chore.1] hbase.ChoreService: Chore [ScheduledChore: Name: CompactionChecker Period: 10000 Unit: MILLISECONDS] is enabled. 2020-11-17 14:38:24,244 WARN [regionserver/cvp21:16201] util.Sleeper: We slept 18138ms instead of 5000ms, this is likely due to a long garbage collecting pause and it's usually bad, see http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired 2020-11-17 14:38:24,491 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:24,493 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:24,497 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:24,501 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:24,502 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:24,535 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:24,538 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:24,495 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: Chore: CompactionChecker missed its start time 2020-11-17 14:38:24,498 INFO [regionserver/cvp21:16201.Chore.2] hbase.ChoreService: Chore [ScheduledChore: Name: MemstoreFlusherChore Period: 10000 Unit: MILLISECONDS] is enabled. 2020-11-17 14:38:24,598 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: Chore: MemstoreFlusherChore missed its start time 2020-11-17 14:38:25,119 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623873929 with entries=2278, filesize=16.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623904487 2020-11-17 14:38:25,120 WARN [Close-WAL-Writer-533] wal.AbstractProtobufLogWriter: Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:424) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:513) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:220) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(AsyncProtobufLogWriter.java:176) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:211) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:235) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:153) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$6(AsyncFSWAL.java:672) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:25,122 WARN [Close-WAL-Writer-533] wal.AsyncProtobufLogWriter: normal close failed, try recover java.lang.IllegalStateException: should call flush first before calling close at org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkState(Preconditions.java:508) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:518) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:569) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:154) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$6(AsyncFSWAL.java:672) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:25,130 INFO [Close-WAL-Writer-533] util.FSHDFSUtils: Recover lease on dfs file /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623873929 2020-11-17 14:38:25,134 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623888414","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BJPE15500897\\ \u003cTRUNCATED\u003e","processingtimems":16718,"client":"192.168.2.23:38618","queuetimems":81,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:38:25,135 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623887858","responsesize":"625","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 101 action(s) and 1st row key\u003d\\x02\\xCD\\x04\\x17\\x94\\xC4\\x06Kernel\\xC4\\x04pro \u003cTRUNCATED\u003e","processingtimems":17276,"client":"192.168.2.21:57452","queuetimems":3,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":101} 2020-11-17 14:38:25,140 INFO [Close-WAL-Writer-533] util.FSHDFSUtils: Failed to recover lease, attempt=0 on file=/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623873929 after 10ms 2020-11-17 14:38:25,167 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623888415","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x97\\xC4\\x07Devices\\xC4 9B5A34985C681B4 \u003cTRUNCATED\u003e","processingtimems":16719,"client":"192.168.2.23:38618","queuetimems":82,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:38:25,173 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623888536","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":16637,"client":"192.168.2.22:47114","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:38:25,174 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623888414","responsesize":"18","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 2 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4 AFD56A36BBBAB77 \u003cTRUNCATED\u003e","processingtimems":16759,"client":"192.168.2.23:38618","queuetimems":81,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 14:38:25,191 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623888664","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":16476,"client":"192.168.2.23:38618","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:38:25,251 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623889649","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":15601,"client":"192.168.2.23:38618","queuetimems":23,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:38:25,252 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623888007","responsesize":"1696","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 257 action(s) and 1st row key\u003d\\x07\\xCD\\x06\\x14\\x98\\xC4\\x05Sysdb\\xC4\\x08hard \u003cTRUNCATED\u003e","processingtimems":17245,"client":"192.168.2.23:38072","queuetimems":13,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":257} 2020-11-17 14:38:25,260 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623889706","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\" \u003cTRUNCATED\u003e","processingtimems":15553,"client":"192.168.2.23:38618","queuetimems":79,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:38:25,310 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623888598","responsesize":"32","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 4 action(s) and 1st row key\u003d\\x02\\xCD\\x06\\x04\\x98\\xC4\\x05Sysdb\\xC4\\x0Benviro \u003cTRUNCATED\u003e","processingtimems":16712,"client":"192.168.2.23:38072","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":4} 2020-11-17 14:38:25,311 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623888379","responsesize":"34","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 4 action(s) and 1st row key\u003d\\x0B\\x06\\x97\\xC4\\x07Devices\\xC4 239840A7260E915 \u003cTRUNCATED\u003e","processingtimems":16932,"client":"192.168.2.23:38618","queuetimems":48,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":4} 2020-11-17 14:38:25,323 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623888045","responsesize":"2966","method":"Multi","param":"region\u003d aeris_v2,\\x06\\xCD\\x04\\x16\\x97\\xC4\\x05Smash\\xC4\\x07routing\\xC4\\x03pim\\xC4\\x0Asparsemode\\xC4\\x06status\\xC4\\x07default\\xC4\\x09routeIntf,160496158 \u003cTRUNCATED\u003e","processingtimems":17207,"client":"192.168.2.22:47084","queuetimems":107,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":439} 2020-11-17 14:38:25,323 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623888415","responsesize":"18","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 2 action(s) and 1st row key\u003d\\x04\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BJPE14192391\\ \u003cTRUNCATED\u003e","processingtimems":16878,"client":"192.168.2.23:38618","queuetimems":82,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 14:38:25,317 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623888489","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":16649,"client":"192.168.2.21:57988","queuetimems":4,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:38:25,353 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623888414","responsesize":"18","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 2 action(s) and 1st row key\u003d\\x04\\x06\\x97\\xC4\\x07Devices\\xC4 996B2ADABF88A40 \u003cTRUNCATED\u003e","processingtimems":16794,"client":"192.168.2.23:38618","queuetimems":80,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 14:38:25,424 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623888415","responsesize":"505","method":"Multi","param":"region\u003d aeris_v2,\\x11\\xCD\\x04\\x0C\\x96\\xC4\\x05Smash\\xC4\\x08routing6\\xC4\\x03pim\\xC4\\x06status\\xC4\\x07default\\xC4\\x14intfSecondaryAddress,1604670627883.8 \u003cTRUNCATED\u003e","processingtimems":17000,"client":"192.168.2.21:57452","queuetimems":3,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":81} 2020-11-17 14:38:29,153 INFO [Close-WAL-Writer-533] util.FSHDFSUtils: Recovered lease, attempt=1 on file=/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623873929 after 4023ms 2020-11-17 14:38:31,225 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1079 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:31,239 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1093 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:31,398 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:38:31,412 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:38:33,222 INFO [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1019ms GC pool 'G1 Young Generation' had collection(s): count=1 time=977ms 2020-11-17 14:38:33,237 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1051 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:35,086 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 577 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:35,086 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 570 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:35,086 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 566 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:35,098 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 547 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:35,099 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 505 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:36,560 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 857 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:36,561 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 855 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:36,561 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 844 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:36,561 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 836 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:39,525 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 889 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:39,525 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 888 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:39,527 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 888 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:39,528 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 884 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:40,243 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:38:40,572 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:38:43,191 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 885 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:43,195 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 869 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:38:53,558 INFO [regionserver/cvp21:16201.Chore.3] hbase.ChoreService: Chore [ScheduledChore: Name: CompactionChecker Period: 10000 Unit: MILLISECONDS] is enabled. 2020-11-17 14:38:53,578 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: Chore: CompactionChecker missed its start time 2020-11-17 14:38:53,584 INFO [regionserver/cvp21:16201.Chore.3] hbase.ChoreService: Chore [ScheduledChore: Name: MemstoreFlusherChore Period: 10000 Unit: MILLISECONDS] is enabled. 2020-11-17 14:38:53,584 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: Chore: MemstoreFlusherChore missed its start time 2020-11-17 14:38:53,684 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:38:54,150 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:54,171 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:54,172 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:54,176 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:54,177 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:54,178 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:54,179 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:54,179 WARN [AsyncFSWAL-0] wal.AsyncFSWAL: sync failed java.io.IOException: Timeout(10000ms) waiting for response at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.lambda$userEventTriggered$4(FanOutOneBlockAsyncDFSOutput.java:303) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.failed(FanOutOneBlockAsyncDFSOutput.java:236) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.access$300(FanOutOneBlockAsyncDFSOutput.java:99) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput$AckHandler.userEventTriggered(FanOutOneBlockAsyncDFSOutput.java:302) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:108) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:366) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelIdle(IdleStateHandler.java:371) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:494) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:466) at org.apache.hbase.thirdparty.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:333) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:54,375 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605623923654","responsesize":"12","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x04tags\\xC4\\x06labels\\xC4\\x07d \u003cTRUNCATED\u003e","processingtimems":10712,"client":"192.168.2.21:57988","queuetimems":5,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":0} 2020-11-17 14:38:54,450 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623904487 with entries=5431, filesize=32.48 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623934173 2020-11-17 14:38:54,454 WARN [Close-WAL-Writer-533] wal.AbstractProtobufLogWriter: Failed to write trailer, non-fatal, continuing... java.io.IOException: stream already broken at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush0(FanOutOneBlockAsyncDFSOutput.java:424) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.flush(FanOutOneBlockAsyncDFSOutput.java:513) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.lambda$writeWALTrailerAndMagic$3(AsyncProtobufLogWriter.java:220) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(AsyncProtobufLogWriter.java:176) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeWALTrailerAndMagic(AsyncProtobufLogWriter.java:211) at org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.writeWALTrailer(AbstractProtobufLogWriter.java:235) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:153) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$6(AsyncFSWAL.java:672) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:54,496 WARN [Close-WAL-Writer-533] wal.AsyncProtobufLogWriter: normal close failed, try recover java.lang.IllegalStateException: should call flush first before calling close at org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkState(Preconditions.java:508) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.endBlock(FanOutOneBlockAsyncDFSOutput.java:518) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.close(FanOutOneBlockAsyncDFSOutput.java:569) at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:154) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$6(AsyncFSWAL.java:672) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-11-17 14:38:54,497 INFO [Close-WAL-Writer-533] util.FSHDFSUtils: Recover lease on dfs file /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623904487 2020-11-17 14:38:54,506 INFO [Close-WAL-Writer-533] util.FSHDFSUtils: Failed to recover lease, attempt=0 on file=/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623904487 after 9ms 2020-11-17 14:38:54,552 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623923768","responsesize":"18","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 2 action(s) and 1st row key\u003d\\x02\\x06\\x99\\xC4\\x07Devices\\xC4\\x0BJAS17330073\\ \u003cTRUNCATED\u003e","processingtimems":10783,"client":"192.168.2.23:38618","queuetimems":2,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 14:38:54,562 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623923910","responsesize":"10","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\xCD\\x03\\xFA\\x96\\xC4\\x05Sysdb\\xC4\\x0Benviro \u003cTRUNCATED\u003e","processingtimems":10651,"client":"192.168.2.21:57452","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 14:38:54,563 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623923485","responsesize":"462","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 74 action(s) and 1st row key\u003d\\x04\\xCD\\x06\\x14\\x97\\xC4\\x05Sysdb\\xC4\\x08hardw \u003cTRUNCATED\u003e","processingtimems":11073,"client":"192.168.2.23:38072","queuetimems":30,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":74} 2020-11-17 14:38:54,600 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623923770","responsesize":"18","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 2 action(s) and 1st row key\u003d\\x09\\x06\\x99\\xC4\\x07Devices\\xC4\\x0BJAS17330073\\ \u003cTRUNCATED\u003e","processingtimems":10780,"client":"192.168.2.23:38618","queuetimems":4,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 14:38:54,788 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623924137","responsesize":"1000","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 158 action(s) and 1st row key\u003d\\x09\\xCD\\x06\\x17\\x98\\xC4\\x05Sysdb\\xC4\\x0Bl2di \u003cTRUNCATED\u003e","processingtimems":10650,"client":"192.168.2.23:38072","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":158} 2020-11-17 14:38:54,814 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623923880","responsesize":"1043","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 164 action(s) and 1st row key\u003d\\x04#\\x98\\xC4\\x05Sysdb\\xC4\\x0Bl2discovery\\xC4 \u003cTRUNCATED\u003e","processingtimems":10934,"client":"192.168.2.22:47084","queuetimems":11,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":164} 2020-11-17 14:38:54,839 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623923811","responsesize":"46","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 6 action(s) and 1st row key\u003d\\x04\\xCD\\x06\\x0E\\x98\\xC4\\x05Sysdb\\xC4\\x0Benviro \u003cTRUNCATED\u003e","processingtimems":11027,"client":"192.168.2.22:47084","queuetimems":2,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":6} 2020-11-17 14:38:54,876 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605623923921","responsesize":"62","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 9 action(s) and 1st row key\u003d\\x02\\x13\\x94\\xC4\\x05Smash\\xC4\\x07routing\\xC4\\x0 \u003cTRUNCATED\u003e","processingtimems":10952,"client":"192.168.2.22:47084","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":9} 2020-11-17 14:38:58,516 INFO [Close-WAL-Writer-533] util.FSHDFSUtils: Recovered lease, attempt=1 on file=/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623904487 after 4019ms 2020-11-17 14:39:06,292 INFO [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 7623ms GC pool 'G1 Young Generation' had collection(s): count=1 time=7555ms 2020-11-17 14:39:06,409 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 7806 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:39:06,456 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 7853 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:39:06,456 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 7851 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:39:06,456 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 7850 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:39:06,457 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 7850 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:39:06,457 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 7846 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK]] 2020-11-17 14:39:06,481 INFO [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:39:14,035 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:39:14,968 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=101.91 MB heapSize=128.03 MB 2020-11-17 14:39:15,077 INFO [Thread-4567] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:39:16,243 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=101.91 MB at sequenceid=261242126 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/94092c2bcbc04d8dbb93e09fc1c4e894 2020-11-17 14:39:16,269 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/94092c2bcbc04d8dbb93e09fc1c4e894, entries=366190, sequenceid=261242126, filesize=6.7 M 2020-11-17 14:39:16,283 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~101.91 MB/106862227, heapSize ~128.03 MB/134251880, currentSize=835.59 KB/855642 for 06f7ca9534710e37d90d9f8345864259 in 1315ms, sequenceid=261242126, compaction requested=true 2020-11-17 14:39:16,285 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 14:39:16,285 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/29ac6b53e458492691bd7afd6f401320, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/7864c3ce89364e559c3bfe83b25fdbf0, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/94092c2bcbc04d8dbb93e09fc1c4e894] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=78.3 M 2020-11-17 14:39:16,361 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:39:16,366 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:39:16,390 INFO [Thread-4569] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:39:17,084 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:39:22,135 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.01 MB heapSize=128.14 MB 2020-11-17 14:39:22,201 INFO [Thread-4571] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:39:23,153 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.01 MB at sequenceid=252682146 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/8caea6386404400ab07d5aece7face27 2020-11-17 14:39:23,160 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:39:23,183 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/8caea6386404400ab07d5aece7face27, entries=316603, sequenceid=252682146, filesize=6.3 M 2020-11-17 14:39:23,185 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.01 MB/111158168, heapSize ~128.14 MB/134365056, currentSize=1.38 MB/1444209 for 82421ce8c5c8b48c573baee83caa8305 in 1050ms, sequenceid=252682146, compaction requested=true 2020-11-17 14:39:23,185 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 14:39:23,186 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/43b6bfe7240d4b72ab79dc668ccf478a, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/66455d2901fc4fdd872fc772b00171ef, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/e73ce9f38db74c5788476ca8b885924c, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/8caea6386404400ab07d5aece7face27] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=227.7 M 2020-11-17 14:39:23,254 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:39:23,260 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:39:23,264 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:39:23,296 INFO [Thread-4573] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:39:36,029 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=117.20 MB, max=1.83 GB, blockCount=45783, accesses=43656725, hits=36908330, hitRatio=84.54%, , cachingAccesses=36962609, cachingHits=36900925, cachingHitsRatio=99.83%, evictions=5456, evicted=0, evictedPerRun=0.0 2020-11-17 14:39:42,250 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:39:47,654 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#1019 average throughput is 35.49 MB/second, slept 20 time(s) and total slept time is 18098 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 14:39:47,775 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 25988e5808e943cab933315e9c9e63b6(size=78.5 M), total size for store is 55.7 G. This selection was in queue for 0sec, and took 31sec to execute. 2020-11-17 14:39:47,775 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605623956283; duration=31sec 2020-11-17 14:39:49,159 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.49 MB heapSize=128.91 MB 2020-11-17 14:39:49,190 INFO [Thread-4576] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:39:50,076 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.49 MB at sequenceid=264519776 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/c5337b67b5f34c818603e91a06b86395 2020-11-17 14:39:50,105 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c5337b67b5f34c818603e91a06b86395, entries=394787, sequenceid=264519776, filesize=7.8 M 2020-11-17 14:39:50,107 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.49 MB/105367025, heapSize ~128.91 MB/135177168, currentSize=3.06 MB/3204977 for e5dc5c620be19b02666a19c280ff582e in 948ms, sequenceid=264519776, compaction requested=true 2020-11-17 14:39:58,407 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623934173 with entries=17634, filesize=128.91 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623998243 2020-11-17 14:40:09,507 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:12,776 INFO [DataStreamer for file /hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/31341e6999b74f4fb9b285d3051d9464] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:16,136 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:24,375 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:24,557 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:32,926 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=85.71 MB heapSize=128.00 MB 2020-11-17 14:40:32,966 INFO [Thread-4581] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:33,683 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:34,135 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:34,138 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:34,140 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:34,219 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:34,410 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:34,581 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=85.71 MB at sequenceid=263087722 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp/e/a18639b62a8e4f9ba12061fb2bb4b244 2020-11-17 14:40:34,618 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/a18639b62a8e4f9ba12061fb2bb4b244, entries=580216, sequenceid=263087722, filesize=8.5 M 2020-11-17 14:40:34,619 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~85.71 MB/89874269, heapSize ~128.00 MB/134218072, currentSize=50.37 KB/51575 for 73aae5715542e1a5370f7540d2117abc in 1693ms, sequenceid=263087722, compaction requested=true 2020-11-17 14:40:34,622 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 14:40:34,622 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/efccc521c2c24444a85fb840360a390e, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/9e3acc949d64427d9abafb1868cf7042, hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/e/a18639b62a8e4f9ba12061fb2bb4b244] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/73aae5715542e1a5370f7540d2117abc/.tmp, totalSize=81.6 M 2020-11-17 14:40:34,695 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:34,699 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:34,733 INFO [Thread-4583] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:35,192 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:35,368 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:38,544 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:41,064 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:41,067 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:41,070 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:41,504 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:41,891 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:41,892 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:42,948 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#1021 average throughput is 47.38 MB/second, slept 71 time(s) and total slept time is 40463 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 14:40:43,077 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 4 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 31341e6999b74f4fb9b285d3051d9464(size=225.6 M), total size for store is 11.6 G. This selection was in queue for 0sec, and took 1mins, 19sec to execute. 2020-11-17 14:40:43,077 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605623963185; duration=1mins, 19sec 2020-11-17 14:40:47,642 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:48,357 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:48,360 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:49,961 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:40:50,985 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 73aae5715542e1a5370f7540d2117abc#e#compaction#1024 average throughput is 45.91 MB/second, slept 12 time(s) and total slept time is 4936 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 14:40:51,059 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 73aae5715542e1a5370f7540d2117abc into 2c8729161034444a8eb34d376b03b302(size=81.9 M), total size for store is 12.8 G. This selection was in queue for 0sec, and took 16sec to execute. 2020-11-17 14:40:51,059 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., storeName=e, priority=11, startTime=1605624034619; duration=16sec 2020-11-17 14:40:58,666 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:41:03,468 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:41:12,789 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623998243 with entries=23804, filesize=128.25 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624072754 2020-11-17 14:41:35,980 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:41:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 66268 ns. 2020-11-17 14:41:38,710 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: nonceCleaner average execution time: 383179 ns. 2020-11-17 14:41:38,710 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 363915 ns. 2020-11-17 14:41:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 70550 ns. 2020-11-17 14:41:51,193 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:41:54,703 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:42:05,335 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=87.44 MB heapSize=128.01 MB 2020-11-17 14:42:05,373 INFO [Thread-4590] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:42:06,679 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=87.44 MB at sequenceid=292795434 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/.tmp/e/632a9bef50c3446db99e3697846863bc 2020-11-17 14:42:06,689 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:42:06,730 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/bdaa31a63b7f3d06d9909b7560333e79/e/632a9bef50c3446db99e3697846863bc, entries=562382, sequenceid=292795434, filesize=8.2 M 2020-11-17 14:42:06,733 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~87.44 MB/91689456, heapSize ~128.01 MB/134231800, currentSize=75.49 KB/77303 for bdaa31a63b7f3d06d9909b7560333e79 in 1398ms, sequenceid=292795434, compaction requested=true 2020-11-17 14:42:08,327 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.92 MB heapSize=128.14 MB 2020-11-17 14:42:08,352 INFO [Thread-4592] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:42:08,982 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.92 MB at sequenceid=252686842 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/f11099b365df4db5a3a1cb2feebe0ffc 2020-11-17 14:42:09,003 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/f11099b365df4db5a3a1cb2feebe0ffc, entries=304109, sequenceid=252686842, filesize=6.3 M 2020-11-17 14:42:09,009 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~106.92 MB/112116059, heapSize ~128.14 MB/134363896, currentSize=370.74 KB/379637 for 82421ce8c5c8b48c573baee83caa8305 in 682ms, sequenceid=252686842, compaction requested=true 2020-11-17 14:42:14,545 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionChecker average execution time: 9308 ns. 2020-11-17 14:42:14,545 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 28286 ns. 2020-11-17 14:42:16,006 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:42:16,800 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624072754 with entries=19736, filesize=128.57 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624136777 2020-11-17 14:42:16,801 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622495340 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605622495340 2020-11-17 14:42:16,812 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622580077 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605622580077 2020-11-17 14:42:16,824 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622684158 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605622684158 2020-11-17 14:42:16,831 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622766306 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605622766306 2020-11-17 14:42:16,837 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622869434 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605622869434 2020-11-17 14:42:16,843 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605622952192 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605622952192 2020-11-17 14:42:16,848 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623056623 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605623056623 2020-11-17 14:42:16,855 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623129094 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605623129094 2020-11-17 14:42:16,862 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623220852 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605623220852 2020-11-17 14:42:16,869 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623287962 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605623287962 2020-11-17 14:42:16,885 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623354918 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605623354918 2020-11-17 14:42:16,894 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623421078 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605623421078 2020-11-17 14:42:16,902 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623476732 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605623476732 2020-11-17 14:42:21,276 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:42:21,280 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:42:33,071 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:42:35,915 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=102.92 MB heapSize=128.12 MB 2020-11-17 14:42:35,953 INFO [Thread-4594] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:42:36,900 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.92 MB at sequenceid=264526989 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/97ab62d720654b7e91046a7e508ab69f 2020-11-17 14:42:36,922 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/97ab62d720654b7e91046a7e508ab69f, entries=348749, sequenceid=264526989, filesize=7.8 M 2020-11-17 14:42:36,923 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.92 MB/107921784, heapSize ~128.12 MB/134341472, currentSize=779.10 KB/797801 for e5dc5c620be19b02666a19c280ff582e in 1008ms, sequenceid=264526989, compaction requested=true 2020-11-17 14:42:36,924 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 14:42:36,924 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/8247264ead944fdc8921cddc4e2aab2d, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c5337b67b5f34c818603e91a06b86395, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/97ab62d720654b7e91046a7e508ab69f] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=23.4 M 2020-11-17 14:42:36,932 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:42:36,974 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:42:37,009 INFO [Thread-4596] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:42:37,053 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:42:38,751 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 58851 ns. 2020-11-17 14:42:41,993 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#1028 average throughput is 60.19 MB/second, slept 6 time(s) and total slept time is 1817 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 14:42:42,086 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into fa9c5da8922e44728cb63de294425840(size=24.4 M), total size for store is 50.2 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 14:42:42,087 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605624156924; duration=5sec 2020-11-17 14:42:48,688 INFO [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:42:54,095 INFO [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:43:02,426 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:43:09,330 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:43:19,656 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:43:33,602 INFO [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:43:36,601 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624136777 with entries=26367, filesize=128.02 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624216558 2020-11-17 14:43:42,271 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:43:55,217 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:44:03,104 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=113.56 MB, max=1.83 GB, blockCount=45854, accesses=44583284, hits=37757179, hitRatio=84.69%, , cachingAccesses=37811571, cachingHits=37749729, cachingHitsRatio=99.84%, evictions=5486, evicted=0, evictedPerRun=0.0 2020-11-17 14:44:59,076 INFO [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:45:04,826 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624216558 with entries=35092, filesize=128.14 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624304785 2020-11-17 14:45:29,745 INFO [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:45:44,158 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=106.97 MB heapSize=128.35 MB 2020-11-17 14:45:44,202 INFO [Thread-4606] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:45:45,000 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=106.97 MB at sequenceid=252693146 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/f684a12475a34ef98695ea7c4a6df39c 2020-11-17 14:45:45,006 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:45:45,030 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/f684a12475a34ef98695ea7c4a6df39c, entries=306004, sequenceid=252693146, filesize=6.3 M 2020-11-17 14:45:45,033 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~106.97 MB/112167471, heapSize ~128.35 MB/134579880, currentSize=384.95 KB/394188 for 82421ce8c5c8b48c573baee83caa8305 in 874ms, sequenceid=252693146, compaction requested=true 2020-11-17 14:45:59,555 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624304785 with entries=23392, filesize=128.31 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624359515 2020-11-17 14:46:00,964 INFO [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:46:09,230 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=104.60 MB heapSize=128.00 MB 2020-11-17 14:46:09,269 INFO [Thread-4609] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:46:10,067 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=104.60 MB at sequenceid=273308779 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/.tmp/e/c682c705c12b4e918275afc8243fa5a9 2020-11-17 14:46:10,072 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:46:10,088 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/0469ce862509756add4a8cdb00bc465c/e/c682c705c12b4e918275afc8243fa5a9, entries=321752, sequenceid=273308779, filesize=8.4 M 2020-11-17 14:46:10,090 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~104.60 MB/109681698, heapSize ~128.00 MB/134222456, currentSize=26.38 KB/27018 for 0469ce862509756add4a8cdb00bc465c in 860ms, sequenceid=273308779, compaction requested=true 2020-11-17 14:46:21,330 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:46:45,405 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:47:06,471 INFO [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:47:11,229 INFO [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:47:15,972 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=102.09 MB heapSize=128.01 MB 2020-11-17 14:47:16,010 INFO [Thread-4615] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:47:16,813 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.09 MB at sequenceid=261265992 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/2cc372668c2c4c68aa8602fe7bd4df2d 2020-11-17 14:47:16,819 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:47:16,835 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/2cc372668c2c4c68aa8602fe7bd4df2d, entries=364634, sequenceid=261265992, filesize=6.7 M 2020-11-17 14:47:16,837 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~102.09 MB/107045040, heapSize ~128.01 MB/134226552, currentSize=59.39 KB/60815 for 06f7ca9534710e37d90d9f8345864259 in 865ms, sequenceid=261265992, compaction requested=true 2020-11-17 14:47:17,972 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.35 MB heapSize=128.00 MB 2020-11-17 14:47:18,005 INFO [Thread-4617] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:47:18,876 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.35 MB at sequenceid=264541693 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/9ce3a31a4fd142518edbf4272b257460 2020-11-17 14:47:18,899 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/9ce3a31a4fd142518edbf4272b257460, entries=383849, sequenceid=264541693, filesize=7.9 M 2020-11-17 14:47:18,901 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.35 MB/105223708, heapSize ~128.00 MB/134218192, currentSize=195.08 KB/199760 for e5dc5c620be19b02666a19c280ff582e in 929ms, sequenceid=264541693, compaction requested=true 2020-11-17 14:47:23,604 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 23672 ns. 2020-11-17 14:47:23,605 INFO [regionserver/cvp21:16201.Chore.2] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of __test-table_cvp23__,,1598623751841.bac2756ecc315bfd457fb71d81dd3eb5. because __status_family___ has an old edit so flush to free WALs after random delay 118739 ms 2020-11-17 14:47:23,605 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 129818 ns. 2020-11-17 14:47:26,762 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:47:35,867 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624359515 with entries=34270, filesize=128.04 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624455825 2020-11-17 14:47:35,868 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623542268 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605623542268 2020-11-17 14:47:35,877 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605623594499 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605623594499 2020-11-17 14:47:36,023 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:47:38,742 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 300228 ns. 2020-11-17 14:47:38,742 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 215819 ns. 2020-11-17 14:47:38,742 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 71948 ns. 2020-11-17 14:47:38,746 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 44825 ns. 2020-11-17 14:47:41,326 INFO [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:48:33,361 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624455825 with entries=22144, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624513339 2020-11-17 14:48:38,665 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 57296 ns. 2020-11-17 14:49:05,087 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=107.00 MB heapSize=128.44 MB 2020-11-17 14:49:05,113 INFO [Thread-4622] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:49:05,703 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.00 MB at sequenceid=252699214 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/688561daaa7144f6a8dfc615916d28a1 2020-11-17 14:49:05,710 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:49:05,728 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/688561daaa7144f6a8dfc615916d28a1, entries=306923, sequenceid=252699214, filesize=6.3 M 2020-11-17 14:49:05,729 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~107.00 MB/112196707, heapSize ~128.44 MB/134679968, currentSize=14.99 KB/15349 for 82421ce8c5c8b48c573baee83caa8305 in 642ms, sequenceid=252699214, compaction requested=true 2020-11-17 14:49:05,730 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 14:49:05,730 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/f11099b365df4db5a3a1cb2feebe0ffc, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/f684a12475a34ef98695ea7c4a6df39c, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/688561daaa7144f6a8dfc615916d28a1] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=18.9 M 2020-11-17 14:49:05,765 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:49:05,770 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:49:05,793 INFO [Thread-4624] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:49:11,222 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#1034 average throughput is 58.92 MB/second, slept 6 time(s) and total slept time is 2532 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 14:49:11,302 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into fe7ad479f6c94fb59bdcb992609843b3(size=18.8 M), total size for store is 11.6 G. This selection was in queue for 0sec, and took 5sec to execute. 2020-11-17 14:49:11,302 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605624545729; duration=5sec 2020-11-17 14:49:22,344 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing bac2756ecc315bfd457fb71d81dd3eb5 1/1 column families, dataSize=256 B heapSize=656 B 2020-11-17 14:49:22,365 INFO [Thread-4627] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:49:22,434 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=256 B at sequenceid=2038 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/__test-table_cvp23__/bac2756ecc315bfd457fb71d81dd3eb5/.tmp/__status_family___/98ce9d2f5fca410faf83eb63fbcba177 2020-11-17 14:49:22,440 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:49:22,457 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/__test-table_cvp23__/bac2756ecc315bfd457fb71d81dd3eb5/__status_family___/98ce9d2f5fca410faf83eb63fbcba177, entries=2, sequenceid=2038, filesize=5.1 K 2020-11-17 14:49:22,461 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~256 B/256, heapSize ~640 B/640, currentSize=0 B/0 for bac2756ecc315bfd457fb71d81dd3eb5 in 116ms, sequenceid=2038, compaction requested=false 2020-11-17 14:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=112.63 MB, max=1.83 GB, blockCount=45880, accesses=44714081, hits=37882757, hitRatio=84.72%, , cachingAccesses=37937172, cachingHits=37875304, cachingHitsRatio=99.84%, evictions=5516, evicted=0, evictedPerRun=0.0 2020-11-17 14:49:46,307 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624513339 with entries=27096, filesize=128.20 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624586282 2020-11-17 14:49:46,307 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605620802496 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605620802496 2020-11-17 14:49:46,316 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Archiving hdfs://mycluster/hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605621341908 to hdfs://mycluster/hbase/oldWALs/cvp21%2C16201%2C1605569374704.1605621341908 2020-11-17 14:50:40,155 INFO [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:50:52,963 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=97.85 MB heapSize=128.22 MB 2020-11-17 14:50:53,004 INFO [Thread-4633] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:50:53,820 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=97.85 MB at sequenceid=264552747 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/c04ea72c33c348b1b9d3768facf48678 2020-11-17 14:50:53,826 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:50:53,843 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c04ea72c33c348b1b9d3768facf48678, entries=422345, sequenceid=264552747, filesize=8.0 M 2020-11-17 14:50:53,845 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~97.85 MB/102597934, heapSize ~128.22 MB/134447784, currentSize=788.01 KB/806922 for e5dc5c620be19b02666a19c280ff582e in 881ms, sequenceid=264552747, compaction requested=true 2020-11-17 14:50:53,845 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 14:50:53,846 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/fa9c5da8922e44728cb63de294425840, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/9ce3a31a4fd142518edbf4272b257460, hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/c04ea72c33c348b1b9d3768facf48678] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp, totalSize=40.3 M 2020-11-17 14:50:53,888 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:50:53,893 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:50:53,925 INFO [Thread-4635] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:50:55,174 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=107.63 MB heapSize=128.00 MB 2020-11-17 14:50:55,214 INFO [Thread-4637] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:50:56,105 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.63 MB at sequenceid=273704948 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/.tmp/e/61a6fd4670de4381aea178dc7d9dadbf 2020-11-17 14:50:56,130 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:50:56,161 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/fc93f6dcc564341dcc983bf30971f73d/e/61a6fd4670de4381aea178dc7d9dadbf, entries=279715, sequenceid=273704948, filesize=8.4 M 2020-11-17 14:50:56,163 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~107.63 MB/112857512, heapSize ~128.00 MB/134217960, currentSize=23.72 KB/24286 for fc93f6dcc564341dcc983bf30971f73d in 989ms, sequenceid=273704948, compaction requested=true 2020-11-17 14:51:02,375 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: e5dc5c620be19b02666a19c280ff582e#e#compaction#1037 average throughput is 58.97 MB/second, slept 9 time(s) and total slept time is 3060 ms. 0 active operations remaining, total limit is 57.69 MB/second 2020-11-17 14:51:02,434 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of e5dc5c620be19b02666a19c280ff582e into bde53402ed6046878a2a5f8aeab540c9(size=41.5 M), total size for store is 50.2 G. This selection was in queue for 0sec, and took 8sec to execute. 2020-11-17 14:51:02,435 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., storeName=e, priority=10, startTime=1605624653845; duration=8sec 2020-11-17 14:51:04,117 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624586282 with entries=29145, filesize=128.41 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624664095 2020-11-17 14:52:05,816 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624664095 with entries=23115, filesize=128.01 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624725777 2020-11-17 14:52:06,379 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=107.11 MB heapSize=128.56 MB 2020-11-17 14:52:06,402 INFO [Thread-4643] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:52:06,983 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=107.11 MB at sequenceid=252704914 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/d8b33fccd0564f0dbc10d3a9c04e0361 2020-11-17 14:52:07,002 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:52:07,024 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d8b33fccd0564f0dbc10d3a9c04e0361, entries=307060, sequenceid=252704914, filesize=6.3 M 2020-11-17 14:52:07,025 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~107.11 MB/112309391, heapSize ~128.56 MB/134803712, currentSize=1.14 MB/1194170 for 82421ce8c5c8b48c573baee83caa8305 in 646ms, sequenceid=252704914, compaction requested=true 2020-11-17 14:52:33,584 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 36738 ns. 2020-11-17 14:52:33,584 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 11363 ns. 2020-11-17 14:52:39,087 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 56358 ns. 2020-11-17 14:53:18,097 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:53:18,100 INFO [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:53:18,571 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624725777 with entries=26914, filesize=130.36 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624798504 2020-11-17 14:53:32,530 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=102.58 MB heapSize=128.19 MB 2020-11-17 14:53:32,578 INFO [Thread-4648] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:53:33,365 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=102.58 MB at sequenceid=261285807 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp/e/0a586e3f5a8740b88822de9a732fb722 2020-11-17 14:53:33,371 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:53:33,392 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/0a586e3f5a8740b88822de9a732fb722, entries=362421, sequenceid=261285807, filesize=6.6 M 2020-11-17 14:53:33,394 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~102.58 MB/107558981, heapSize ~128.19 MB/134417088, currentSize=911.11 KB/932977 for 06f7ca9534710e37d90d9f8345864259 in 864ms, sequenceid=261285807, compaction requested=true 2020-11-17 14:53:33,395 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 14:53:33,395 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/25988e5808e943cab933315e9c9e63b6, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/2cc372668c2c4c68aa8602fe7bd4df2d, hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/e/0a586e3f5a8740b88822de9a732fb722] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/06f7ca9534710e37d90d9f8345864259/.tmp, totalSize=91.9 M 2020-11-17 14:53:33,442 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:53:33,448 INFO [regionserver/cvp21:16201-shortCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:53:33,474 INFO [Thread-4650] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 245280 ns. 2020-11-17 14:53:38,710 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 69521 ns. 2020-11-17 14:53:38,710 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 62513 ns. 2020-11-17 14:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 191134 ns. 2020-11-17 14:53:50,115 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.69 MB heapSize=128.49 MB 2020-11-17 14:53:50,324 INFO [Thread-4653] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:53:50,963 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.69 MB at sequenceid=252708041 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/935456381b994d6ca7556d56398f116b 2020-11-17 14:53:50,986 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:53:51,011 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/935456381b994d6ca7556d56398f116b, entries=327501, sequenceid=252708041, filesize=6.1 M 2020-11-17 14:53:51,013 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~105.69 MB/110824526, heapSize ~128.50 MB/134741040, currentSize=1.32 MB/1384122 for 82421ce8c5c8b48c573baee83caa8305 in 898ms, sequenceid=252708041, compaction requested=true 2020-11-17 14:53:51,014 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HRegion: Starting compaction of e in aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 14:53:51,014 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Starting compaction of [hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/fe7ad479f6c94fb59bdcb992609843b3, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/d8b33fccd0564f0dbc10d3a9c04e0361, hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/935456381b994d6ca7556d56398f116b] into tmpdir=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp, totalSize=31.1 M 2020-11-17 14:53:51,047 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:53:51,049 INFO [regionserver/cvp21:16201-longCompactions-0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:53:51,072 INFO [Thread-4655] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:53:56,938 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624798504 with entries=13716, filesize=128.30 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624836916 2020-11-17 14:54:00,879 INFO [regionserver/cvp21:16201-shortCompactions-0] throttle.PressureAwareThroughputController: 06f7ca9534710e37d90d9f8345864259#e#compaction#1041 average throughput is 47.93 MB/second, slept 24 time(s) and total slept time is 13275 ms. 1 active operations remaining, total limit is 61.54 MB/second 2020-11-17 14:54:00,983 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 06f7ca9534710e37d90d9f8345864259 into 6c9a402aa1994777bd6162d7e5003de3(size=92.0 M), total size for store is 55.7 G. This selection was in queue for 0sec, and took 27sec to execute. 2020-11-17 14:54:00,983 INFO [regionserver/cvp21:16201-shortCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., storeName=e, priority=10, startTime=1605624813394; duration=27sec 2020-11-17 14:54:05,154 INFO [regionserver/cvp21:16201-longCompactions-0] throttle.PressureAwareThroughputController: 82421ce8c5c8b48c573baee83caa8305#e#compaction#1043 average throughput is 37.85 MB/second, slept 9 time(s) and total slept time is 8664 ms. 0 active operations remaining, total limit is 61.54 MB/second 2020-11-17 14:54:05,229 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.HStore: Completed compaction of 3 file(s) in e of 82421ce8c5c8b48c573baee83caa8305 into 357b863c16f64d3aba451a1cbfec2d4e(size=31.1 M), total size for store is 11.7 G. This selection was in queue for 0sec, and took 14sec to execute. 2020-11-17 14:54:05,230 INFO [regionserver/cvp21:16201-longCompactions-0] regionserver.CompactSplit: Completed compaction region=aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305., storeName=e, priority=10, startTime=1605624831013; duration=14sec 2020-11-17 14:54:18,139 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=100.67 MB heapSize=128.13 MB 2020-11-17 14:54:18,188 INFO [Thread-4658] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:54:18,940 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=100.67 MB at sequenceid=264563218 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/.tmp/e/fd74854b5b2a4e08965539bb555fcc5e 2020-11-17 14:54:18,947 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:54:18,975 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/e5dc5c620be19b02666a19c280ff582e/e/fd74854b5b2a4e08965539bb555fcc5e, entries=380287, sequenceid=264563218, filesize=8.1 M 2020-11-17 14:54:18,977 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~100.67 MB/105562837, heapSize ~128.13 MB/134355640, currentSize=1.97 MB/2070391 for e5dc5c620be19b02666a19c280ff582e in 838ms, sequenceid=264563218, compaction requested=true 2020-11-17 14:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=113.15 MB, max=1.83 GB, blockCount=45866, accesses=44754628, hits=37885184, hitRatio=84.65%, , cachingAccesses=37939586, cachingHits=37877708, cachingHitsRatio=99.84%, evictions=5546, evicted=0, evictedPerRun=0.0 2020-11-17 14:54:38,857 INFO [regionserver/cvp21:16201.logRoller] wal.AbstractFSWAL: Rolled WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624836916 with entries=15653, filesize=128.65 MB; new WAL /hbase/WALs/cvp21,16201,1605569374704/cvp21%2C16201%2C1605569374704.1605624878801 2020-11-17 14:55:49,171 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=105.56 MB heapSize=128.17 MB 2020-11-17 14:55:49,272 INFO [Thread-4665] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:55:50,221 INFO [MemStoreFlusher.1] regionserver.DefaultStoreFlusher: Flushed memstore data size=105.56 MB at sequenceid=252711506 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/.tmp/e/4f0510a48ac64ece8605fc3168110fa7 2020-11-17 14:55:50,284 INFO [MemStoreFlusher.1] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:55:50,387 INFO [MemStoreFlusher.1] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/82421ce8c5c8b48c573baee83caa8305/e/4f0510a48ac64ece8605fc3168110fa7, entries=324518, sequenceid=252711506, filesize=6.1 M 2020-11-17 14:55:50,422 INFO [MemStoreFlusher.1] regionserver.HRegion: Finished flush of dataSize ~105.56 MB/110682853, heapSize ~128.17 MB/134400888, currentSize=428.70 KB/438991 for 82421ce8c5c8b48c573baee83caa8305 in 1252ms, sequenceid=252711506, compaction requested=true 2020-11-17 14:55:53,967 INFO [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1233ms GC pool 'G1 Young Generation' had collection(s): count=1 time=1237ms 2020-11-17 14:55:53,990 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1338 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:55:53,990 INFO [AsyncFSWAL-0] wal.AbstractFSWAL: Slow sync cost: 1338 ms, current pipeline: [DatanodeInfoWithStorage[192.168.2.21:15010,DS-50cb293a-46f2-4fc3-87a0-55acb999fbc9,DISK], DatanodeInfoWithStorage[192.168.2.22:15010,DS-51b112e1-7790-421c-bc9e-75d82de63b4d,DISK], DatanodeInfoWithStorage[192.168.2.23:15010,DS-43f750d5-b505-4a10-ab93-d84ee01672aa,DISK]] 2020-11-17 14:55:54,697 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=89.83 MB heapSize=128.01 MB 2020-11-17 14:55:54,808 INFO [Thread-4667] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:55:56,757 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed memstore data size=89.83 MB at sequenceid=289026633 (bloomFilter=true), to=hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/.tmp/e/a8951641c0e342c6a3f894071a8320b9 2020-11-17 14:55:56,783 INFO [MemStoreFlusher.0] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 14:55:56,824 INFO [MemStoreFlusher.0] regionserver.HStore: Added hdfs://mycluster/hbase/data/default/aeris_v2/5e5f05205d7e40b0141cd977dd009ba1/e/a8951641c0e342c6a3f894071a8320b9, entries=533744, sequenceid=289026633, filesize=8.6 M 2020-11-17 14:55:56,836 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished flush of dataSize ~89.83 MB/94197986, heapSize ~128.01 MB/134224400, currentSize=218.63 KB/223878 for 5e5f05205d7e40b0141cd977dd009ba1 in 2140ms, sequenceid=289026633, compaction requested=true 2020-11-17 14:56:15,700 WARN [regionserver/cvp21:16201] util.Sleeper: We slept 17760ms instead of 5000ms, this is likely due to a long garbage collecting pause and it's usually bad, see http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired 2020-11-17 14:56:15,709 INFO [regionserver/cvp21:16201.Chore.3] hbase.ChoreService: Chore [ScheduledChore: Name: CompactionChecker Period: 10000 Unit: MILLISECONDS] is enabled. 2020-11-17 14:56:15,757 INFO [regionserver/cvp21:16201.Chore.2] hbase.ChoreService: Chore [ScheduledChore: Name: MemstoreFlusherChore Period: 10000 Unit: MILLISECONDS] is enabled. 2020-11-17 14:56:15,765 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: Chore: MemstoreFlusherChore missed its start time 2020-11-17 14:56:15,766 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: Chore: CompactionChecker missed its start time 2020-11-17 14:56:37,756 INFO [regionserver/cvp21:16201.Chore.1] hbase.ChoreService: Chore [ScheduledChore: Name: MemstoreFlusherChore Period: 10000 Unit: MILLISECONDS] is enabled. 2020-11-17 14:56:37,778 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: Chore: MemstoreFlusherChore missed its start time 2020-11-17 14:56:37,756 WARN [regionserver/cvp21:16201] util.Sleeper: We slept 15883ms instead of 5000ms, this is likely due to a long garbage collecting pause and it's usually bad, see http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired 2020-11-17 14:56:37,779 INFO [regionserver/cvp21:16201.Chore.2] hbase.ChoreService: Chore [ScheduledChore: Name: CompactionChecker Period: 10000 Unit: MILLISECONDS] is enabled. 2020-11-17 14:56:37,779 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: Chore: CompactionChecker missed its start time 2020-11-17 14:57:37,827 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 40012 ns. 2020-11-17 14:57:37,827 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionChecker average execution time: 12991 ns. 2020-11-17 14:58:38,747 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 57823 ns. 2020-11-17 14:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=113.94 MB, max=1.83 GB, blockCount=45852, accesses=44758407, hits=37888963, hitRatio=84.65%, , cachingAccesses=37943365, cachingHits=37881487, cachingHitsRatio=99.84%, evictions=5574, evicted=0, evictedPerRun=0.0 2020-11-17 14:59:38,658 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 111735 ns. 2020-11-17 14:59:38,706 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 228286 ns. 2020-11-17 14:59:38,746 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 63756 ns. 2020-11-17 14:59:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 132752 ns. 2020-11-17 15:01:02,349 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961787","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ16292009\\ \u003cTRUNCATED\u003e","processingtimems":300561,"client":"192.168.2.23:38618","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:01:02,370 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961804","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624\\ \u003cTRUNCATED\u003e","processingtimems":300565,"client":"192.168.2.23:38618","queuetimems":18,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:01:02,474 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961790","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17250690\\ \u003cTRUNCATED\u003e","processingtimems":300684,"client":"192.168.2.23:38618","queuetimems":2,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:01:02,487 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961790","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280689\\ \u003cTRUNCATED\u003e","processingtimems":300697,"client":"192.168.2.23:38618","queuetimems":5,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:01:02,601 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961787","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624\\ \u003cTRUNCATED\u003e","processingtimems":300814,"client":"192.168.2.23:38618","queuetimems":2,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:02:48,682 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 25422 ns. 2020-11-17 15:02:48,682 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 52774 ns. 2020-11-17 15:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=113.94 MB, max=1.83 GB, blockCount=45852, accesses=44758407, hits=37888963, hitRatio=84.65%, , cachingAccesses=37943365, cachingHits=37881487, cachingHitsRatio=99.84%, evictions=5604, evicted=0, evictedPerRun=0.0 2020-11-17 15:04:38,753 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 39052 ns. 2020-11-17 15:05:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 51334 ns. 2020-11-17 15:05:38,711 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 64517 ns. 2020-11-17 15:05:38,711 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 208066 ns. 2020-11-17 15:05:38,711 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: nonceCleaner average execution time: 138172 ns. 2020-11-17 15:06:02,157 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961636","responsesize":"20084","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 48 action(s) and 1st row key\u003d\\x02\\x10\\x98\\xC4\\x05Sysdb\\xC4\\x08hardware\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":600521,"client":"192.168.2.22:47084","queuetimems":12,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":48} 2020-11-17 15:06:02,279 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961787","responsesize":"4970","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 3 action(s) and 1st row key\u003d\\x07\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BHNN20205024\\ \u003cTRUNCATED\u003e","processingtimems":600492,"client":"192.168.2.23:38618","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 15:06:02,389 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961497","responsesize":"5062","method":"Multi","param":"region\u003d aeris_v2,\\x06\\xCD\\x04\\x16\\x97\\xC4\\x05Smash\\xC4\\x07routing\\xC4\\x03pim\\xC4\\x0Asparsemode\\xC4\\x06status\\xC4\\x07default\\xC4\\x09routeIntf,160496158 \u003cTRUNCATED\u003e","processingtimems":600892,"client":"192.168.2.21:57452","queuetimems":5,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":17} 2020-11-17 15:06:02,411 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625262350","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BJPE16063289\\ \u003cTRUNCATED\u003e","processingtimems":300061,"client":"192.168.2.23:38618","queuetimems":300562,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:06:02,431 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625262370","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17250690\\ \u003cTRUNCATED\u003e","processingtimems":300061,"client":"192.168.2.23:38618","queuetimems":300582,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:06:02,485 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961787","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 2 action(s) and 1st row key\u003d\\x07\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ16292009\\ \u003cTRUNCATED\u003e","processingtimems":600698,"client":"192.168.2.23:38618","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 15:06:02,619 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961787","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 2 action(s) and 1st row key\u003d\\x0B\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ18281992\\ \u003cTRUNCATED\u003e","processingtimems":600832,"client":"192.168.2.23:38618","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 15:07:57,801 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionChecker average execution time: 21950 ns. 2020-11-17 15:07:57,802 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 44686 ns. 2020-11-17 15:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=113.94 MB, max=1.83 GB, blockCount=45852, accesses=44758407, hits=37888963, hitRatio=84.65%, , cachingAccesses=37943365, cachingHits=37881487, cachingHitsRatio=99.84%, evictions=5634, evicted=0, evictedPerRun=0.0 2020-11-17 15:09:39,282 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 48692 ns. 2020-11-17 15:11:02,327 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625562279","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17250690\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.23:38618","queuetimems":600490,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:11:02,358 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961789","responsesize":"4973","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 3 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ16292009\\ \u003cTRUNCATED\u003e","processingtimems":900569,"client":"192.168.2.23:38618","queuetimems":3,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 15:11:02,435 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625562391","responsesize":"3313","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 2 action(s) and 1st row key\u003d\\x04\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17451006\\ \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.23:38618","queuetimems":600602,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 15:11:02,495 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625562432","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ18281983\\ \u003cTRUNCATED\u003e","processingtimems":300063,"client":"192.168.2.23:38618","queuetimems":600639,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:11:02,533 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625562487","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BJPE19362489\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.23:38618","queuetimems":600665,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:11:02,653 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961753","responsesize":"13243","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 8 action(s) and 1st row key\u003d\\x04\\xCD\\x06\\x16\\x96\\xC4\\x05Smash\\xC4\\x07routin \u003cTRUNCATED\u003e","processingtimems":900900,"client":"192.168.2.22:47084","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":8} 2020-11-17 15:11:02,789 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625262603","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 2 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624\\ \u003cTRUNCATED\u003e","processingtimems":600186,"client":"192.168.2.23:38618","queuetimems":300814,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 15:11:38,658 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 79832 ns. 2020-11-17 15:11:38,704 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 276565 ns. 2020-11-17 15:11:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 36107 ns. 2020-11-17 15:11:38,710 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: nonceCleaner average execution time: 189377 ns. 2020-11-17 15:12:58,716 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionChecker average execution time: 20711 ns. 2020-11-17 15:12:58,716 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 34561 ns. 2020-11-17 15:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=113.94 MB, max=1.83 GB, blockCount=45852, accesses=44758407, hits=37888963, hitRatio=84.65%, , cachingAccesses=37943365, cachingHits=37881487, cachingHitsRatio=99.84%, evictions=5664, evicted=0, evictedPerRun=0.0 2020-11-17 15:15:38,746 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 46367 ns. 2020-11-17 15:16:02,047 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961759","responsesize":"11592","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 7 action(s) and 1st row key\u003d\\x0B\\xCD\\x06!\\x96\\xC4\\x05Sysdb\\xC4\\x0Benvironme \u003cTRUNCATED\u003e","processingtimems":1200288,"client":"192.168.2.21:57452","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":7} 2020-11-17 15:16:02,056 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961789","responsesize":"11592","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 7 action(s) and 1st row key\u003d\\x0B\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ18295153\\ \u003cTRUNCATED\u003e","processingtimems":1200267,"client":"192.168.2.23:38618","queuetimems":4,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":7} 2020-11-17 15:16:02,254 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625562158","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 2 action(s) and 1st row key\u003d\\x07\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BJPE16063289\\ \u003cTRUNCATED\u003e","processingtimems":600096,"client":"192.168.2.23:38618","queuetimems":600369,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 15:16:02,409 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625862358","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:57988","queuetimems":885642,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:16:02,491 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625862438","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:57988","queuetimems":885737,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:16:02,545 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625862497","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:57988","queuetimems":885782,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:16:02,587 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625862535","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:57988","queuetimems":885817,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:16:02,628 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625262476","responsesize":"9935","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 6 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ16292009\\ \u003cTRUNCATED\u003e","processingtimems":900152,"client":"192.168.2.23:38618","queuetimems":300688,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":6} 2020-11-17 15:16:02,634 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625262489","responsesize":"6627","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 4 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624\\ \u003cTRUNCATED\u003e","processingtimems":900144,"client":"192.168.2.23:38618","queuetimems":300701,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":4} 2020-11-17 15:16:02,701 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625862654","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:57988","queuetimems":885929,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:16:02,842 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625862791","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:57988","queuetimems":886061,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:16:08,994 INFO [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 15:16:09,733 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2020-11-17 15:17:38,658 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 55878 ns. 2020-11-17 15:17:38,714 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 44940 ns. 2020-11-17 15:17:38,747 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 288053 ns. 2020-11-17 15:17:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 222544 ns. 2020-11-17 15:18:08,323 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 28300 ns. 2020-11-17 15:18:08,323 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionChecker average execution time: 25510 ns. 2020-11-17 15:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44821334, hits=37951831, hitRatio=84.67%, , cachingAccesses=38006292, cachingHits=37944355, cachingHitsRatio=99.84%, evictions=5694, evicted=0, evictedPerRun=0.0 2020-11-17 15:20:38,747 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 44495 ns. 2020-11-17 15:21:02,474 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625862327","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 2 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":600146,"client":"192.168.2.21:57988","queuetimems":885615,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 15:21:02,546 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961787","responsesize":"8287","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 5 action(s) and 1st row key\u003d\\x02\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ16292009\\ \u003cTRUNCATED\u003e","processingtimems":1500759,"client":"192.168.2.23:38618","queuetimems":2,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":5} 2020-11-17 15:21:02,569 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625562413","responsesize":"4973","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 3 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4 AFD56A36BBBAB77 \u003cTRUNCATED\u003e","processingtimems":900156,"client":"192.168.2.23:38618","queuetimems":600624,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 15:21:06,996 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626166945","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x94\\xC4\\x0Acompliance\\xC4\\x0BJPE144830 \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.23:59864","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:21:09,919 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626169859","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x95\\xC4\\x0AztpService\\xC4\\x06device\\xC \u003cTRUNCATED\u003e","processingtimems":300060,"client":"192.168.2.21:59818","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:21:10,152 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: Can not complete this request in time, drop it: callId: 164 service: ClientService methodName: Mutate size: 373 connection: 192.168.2.23:59864 deadline: 9223372036854775807 param: region= aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., row=\x0B\x07\x93\xC4\x09snapshots\xC4\x06config\xC4\x0DrequestStatus connection: 192.168.2.23:59864 2020-11-17 15:21:12,096 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626172045","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x90","processingtimems":300051,"client":"192.168.2.21:59818","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:21:23,001 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626182949","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:59818","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:21:23,013 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626182969","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624\\ \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.23:59864","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:21:40,152 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626200106","responsesize":"3313","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 2 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ16292009\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.23:59864","queuetimems":17137,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 15:22:38,745 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 72561 ns. 2020-11-17 15:23:17,779 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 38291 ns. 2020-11-17 15:23:17,779 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionChecker average execution time: 16078 ns. 2020-11-17 15:23:38,710 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 54143 ns. 2020-11-17 15:23:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 251286 ns. 2020-11-17 15:23:38,710 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 204748 ns. 2020-11-17 15:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44821520, hits=37952017, hitRatio=84.67%, , cachingAccesses=38006478, cachingHits=37944541, cachingHitsRatio=99.84%, evictions=5724, evicted=0, evictedPerRun=0.0 2020-11-17 15:26:02,605 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626462548","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280689\\ \u003cTRUNCATED\u003e","processingtimems":300057,"client":"192.168.2.23:47262","queuetimems":68,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:26:02,697 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":19,"starttimems":"1605626462571","responsesize":"3465","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 21 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x0AztpService\\xC4\\x06config\\x \u003cTRUNCATED\u003e","processingtimems":300126,"client":"192.168.2.21:42514","queuetimems":92,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 15:26:02,727 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605624961759","responsesize":"62873","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 38 action(s) and 1st row key\u003d\\x03\\xCC\\xCF\\x96\\xC4\\x05Sysdb\\xC4\\x0Benvironme \u003cTRUNCATED\u003e","processingtimems":1800967,"client":"192.168.2.23:38072","queuetimems":3,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":38} 2020-11-17 15:26:10,202 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: Can not complete this request in time, drop it: callId: 8 service: ClientService methodName: Mutate size: 372 connection: 192.168.2.23:47262 deadline: 9223372036854775807 param: region= aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., row=\x0B\x07\x93\xC4\x09snapshots\xC4\x06config\xC4\x0DrequestStatus connection: 192.168.2.23:47262 2020-11-17 15:26:12,144 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626472098","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:42514","queuetimems":9617,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:26:23,074 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626483014","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300060,"client":"192.168.2.21:42514","queuetimems":15600,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:26:37,056 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626496999","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300057,"client":"192.168.2.21:42514","queuetimems":29584,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:26:38,747 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 51368 ns. 2020-11-17 15:26:40,199 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626500154","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.21:42514","queuetimems":32653,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:28:18,635 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 45190 ns. 2020-11-17 15:28:18,635 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionChecker average execution time: 20212 ns. 2020-11-17 15:28:38,733 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 71829 ns. 2020-11-17 15:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44821669, hits=37952166, hitRatio=84.67%, , cachingAccesses=38006627, cachingHits=37944690, cachingHitsRatio=99.84%, evictions=5754, evicted=0, evictedPerRun=0.0 2020-11-17 15:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 15:29:38,703 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 209479 ns. 2020-11-17 15:29:38,710 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: nonceCleaner average execution time: 234748 ns. 2020-11-17 15:29:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 40359 ns. 2020-11-17 15:31:02,653 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626762608","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x90","processingtimems":300044,"client":"192.168.2.21:50942","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:31:02,751 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626762702","responsesize":"6621","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 4 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:50942","queuetimems":94,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":4} 2020-11-17 15:31:02,782 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626762730","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280689\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.23:55816","queuetimems":27,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:31:02,868 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605625562621","responsesize":"8287","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 5 action(s) and 1st row key\u003d\\x0B\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17250690\\ \u003cTRUNCATED\u003e","processingtimems":1500246,"client":"192.168.2.23:38618","queuetimems":600832,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":5} 2020-11-17 15:31:07,332 INFO [qtp368802354-344] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-17 15:31:09,904 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":21,"starttimems":"1605626169734","responsesize":"32880","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 24 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06config\\xC4 \u003cTRUNCATED\u003e","processingtimems":900170,"client":"192.168.2.21:59818","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 15:31:10,045 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605626469921","responsesize":"4980","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 4 action(s) and 1st row key\u003d\\x09\\x02\\x93\\xC4\\x04oidc\\xC4\\x08sessions\\xC4if8 \u003cTRUNCATED\u003e","processingtimems":600123,"client":"192.168.2.21:42514","queuetimems":7440,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 15:31:10,249 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: Can not complete this request in time, drop it: callId: 14 service: ClientService methodName: Mutate size: 372 connection: 192.168.2.23:55816 deadline: 9223372036854775807 param: region= aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., row=\x0B\x07\x93\xC4\x09snapshots\xC4\x06config\xC4\x0DrequestStatus connection: 192.168.2.23:55816 2020-11-17 15:31:12,192 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626772146","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:50942","queuetimems":9538,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:31:23,105 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626483002","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 2 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":600103,"client":"192.168.2.21:42514","queuetimems":15588,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 15:31:23,132 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626783074","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x94\\xC4\\x0Acompliance\\xC4\\x0BJPE144830 \u003cTRUNCATED\u003e","processingtimems":300058,"client":"192.168.2.23:55816","queuetimems":20465,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:31:37,114 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626797058","responsesize":"13237","method":"Multi","param":"region\u003d aeris_v2,\\x06\\xCD\\x04\\x16\\x97\\xC4\\x05Smash\\xC4\\x07routing\\xC4\\x03pim\\xC4\\x0Asparsemode\\xC4\\x06status\\xC4\\x07default\\xC4\\x09routeIntf,160496158 \u003cTRUNCATED\u003e","processingtimems":300056,"client":"192.168.2.23:55784","queuetimems":34419,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":8} 2020-11-17 15:31:40,331 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626200106","responsesize":"4973","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 3 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624\\ \u003cTRUNCATED\u003e","processingtimems":900225,"client":"192.168.2.23:59864","queuetimems":17137,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 15:31:40,334 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626200106","responsesize":"6627","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 4 action(s) and 1st row key\u003d\\x0B\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BHNN20205024\\ \u003cTRUNCATED\u003e","processingtimems":900228,"client":"192.168.2.23:59864","queuetimems":17137,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":4} 2020-11-17 15:31:40,335 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626800203","responsesize":"28124","method":"Multi","param":"region\u003d aeris_v2,\\x06\\xCD\\x04\\x16\\x97\\xC4\\x05Smash\\xC4\\x07routing\\xC4\\x03pim\\xC4\\x0Asparsemode\\xC4\\x06status\\xC4\\x07default\\xC4\\x09routeIntf,160496158 \u003cTRUNCATED\u003e","processingtimems":300131,"client":"192.168.2.23:55784","queuetimems":22662,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":17} 2020-11-17 15:32:38,746 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 51781 ns. 2020-11-17 15:33:27,990 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 52909 ns. 2020-11-17 15:33:27,990 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionChecker average execution time: 32483 ns. 2020-11-17 15:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44821707, hits=37952204, hitRatio=84.67%, , cachingAccesses=38006665, cachingHits=37944728, cachingHitsRatio=99.84%, evictions=5784, evicted=0, evictedPerRun=0.0 2020-11-17 15:34:38,672 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 76928 ns. 2020-11-17 15:35:38,706 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 192998 ns. 2020-11-17 15:35:38,710 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 51845 ns. 2020-11-17 15:35:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 182282 ns. 2020-11-17 15:36:09,975 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627069907","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300067,"client":"192.168.2.23:36216","queuetimems":7251,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:36:10,094 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627070045","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x94\\xC4\\x0Acompliance\\xC4\\x0BJPE144830 \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.23:36216","queuetimems":7388,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:36:10,385 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: Can not complete this request in time, drop it: callId: 11 service: ClientService methodName: Mutate size: 372 connection: 192.168.2.23:36216 deadline: 9223372036854775807 param: region= aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., row=\x0B\x07\x93\xC4\x09snapshots\xC4\x06config\xC4\x0DrequestStatus connection: 192.168.2.23:36216 2020-11-17 15:36:10,387 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605627070251","responsesize":"3321","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 3 action(s) and 1st row key\u003d\\x02\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300136,"client":"192.168.2.22:47444","queuetimems":7594,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 15:36:12,388 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627072194","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x94\\xC4\\x09BugAlerts\\xC4\\x06update\\xC4 \u003cTRUNCATED\u003e","processingtimems":300193,"client":"192.168.2.22:47444","queuetimems":9537,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:36:23,153 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627083105","responsesize":"3313","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 2 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:59516","queuetimems":20446,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 15:36:23,195 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626182969","responsesize":"8284","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 5 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624\\ \u003cTRUNCATED\u003e","processingtimems":1200226,"client":"192.168.2.23:59864","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":5} 2020-11-17 15:36:23,198 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627083138","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300060,"client":"192.168.2.21:59516","queuetimems":20479,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:36:32,806 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627092755","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:59516","queuetimems":30096,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:36:32,811 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948933 2020-11-17 15:36:32,832 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627092784","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:59516","queuetimems":30125,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:36:32,922 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627092872","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:59516","queuetimems":30212,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:36:37,164 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627097116","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:59516","queuetimems":34457,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:36:40,390 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627100333","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300057,"client":"192.168.2.21:59516","queuetimems":37673,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:36:40,390 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627100339","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:59516","queuetimems":37679,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:36:40,393 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627100339","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x90","processingtimems":300054,"client":"192.168.2.21:59516","queuetimems":37679,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:37:10,400 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605626530156","responsesize":"81060","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 49 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4 01335AA97C749A \u003cTRUNCATED\u003e","processingtimems":900243,"client":"192.168.2.23:47262","queuetimems":67675,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":49} 2020-11-17 15:38:38,238 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 37426 ns. 2020-11-17 15:38:38,238 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 17541 ns. 2020-11-17 15:38:38,746 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 39295 ns. 2020-11-17 15:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44821947, hits=37952444, hitRatio=84.67%, , cachingAccesses=38006905, cachingHits=37944968, cachingHitsRatio=99.84%, evictions=5814, evicted=0, evictedPerRun=0.0 2020-11-17 15:40:38,184 INFO [regionserver/cvp21:16201.Chore.2] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. because e has an old edit so flush to free WALs after random delay 150676 ms 2020-11-17 15:40:38,658 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 77430 ns. 2020-11-17 15:41:10,039 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627369975","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x95\\xC4\\x0AztpService\\xC4\\x06device\\xC \u003cTRUNCATED\u003e","processingtimems":300063,"client":"192.168.2.21:59516","queuetimems":307315,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:41:10,154 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627370096","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300058,"client":"192.168.2.21:59516","queuetimems":307342,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:41:10,438 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627370388","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:59516","queuetimems":307728,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:41:10,442 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627370387","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300055,"client":"192.168.2.21:59516","queuetimems":307633,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:41:12,440 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627372392","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:59516","queuetimems":309631,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:41:23,249 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627383197","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:59516","queuetimems":320442,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:41:23,253 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627383200","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.21:59516","queuetimems":284942,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:41:32,867 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627392816","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:59516","queuetimems":308662,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:41:32,897 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627392834","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300062,"client":"192.168.2.21:59516","queuetimems":308505,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:41:32,985 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627392922","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300063,"client":"192.168.2.21:59516","queuetimems":308454,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:41:37,218 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627397165","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:59516","queuetimems":298781,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:41:38,714 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: nonceCleaner average execution time: 132902 ns. 2020-11-17 15:41:38,714 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 196104 ns. 2020-11-17 15:41:38,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 50119 ns. 2020-11-17 15:41:40,439 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627400394","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.21:59516","queuetimems":302004,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:41:40,441 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627400393","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:59516","queuetimems":302135,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:41:40,451 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627400394","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300056,"client":"192.168.2.21:59516","queuetimems":302050,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:42:07,783 INFO [regionserver/cvp21:16201.Chore.2] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. because e has an old edit so flush to free WALs after random delay 29482 ms 2020-11-17 15:42:10,451 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627430402","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:59516","queuetimems":332043,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:42:37,266 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing bdaa31a63b7f3d06d9909b7560333e79 1/1 column families, dataSize=50.34 MB heapSize=73.87 MB 2020-11-17 15:42:38,354 INFO [regionserver/cvp21:16201.Chore.3] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. because e has an old edit so flush to free WALs after random delay 165094 ms 2020-11-17 15:43:08,860 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=84.59 MB heapSize=125.97 MB 2020-11-17 15:43:18,633 INFO [regionserver/cvp21:16201.Chore.2] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. because e has an old edit so flush to free WALs after random delay 259483 ms 2020-11-17 15:43:47,778 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 166859 ns. 2020-11-17 15:43:47,779 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionChecker average execution time: 23089 ns. 2020-11-17 15:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44821971, hits=37952468, hitRatio=84.67%, , cachingAccesses=38006929, cachingHits=37944992, cachingHitsRatio=99.84%, evictions=5844, evicted=0, evictedPerRun=0.0 2020-11-17 15:44:38,755 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 33514 ns. 2020-11-17 15:45:38,658 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 83324 ns. 2020-11-17 15:46:10,092 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627670039","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:59516","queuetimems":571638,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:46:10,491 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627670440","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:59516","queuetimems":572039,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:46:10,494 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627670443","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:59516","queuetimems":496166,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:46:12,488 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627672441","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:59516","queuetimems":497999,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:46:18,092 INFO [regionserver/cvp21:16201.Chore.3] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. because e has an old edit so flush to free WALs after random delay 35922 ms 2020-11-17 15:46:23,259 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627383156","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 2 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":600102,"client":"192.168.2.21:59516","queuetimems":284898,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 15:46:23,301 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627683254","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:59516","queuetimems":508666,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:46:23,306 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627683253","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:59516","queuetimems":584794,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:46:32,934 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627692869","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300065,"client":"192.168.2.21:59516","queuetimems":568932,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:46:32,959 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627692902","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300057,"client":"192.168.2.21:59516","queuetimems":504523,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:46:33,048 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627692986","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300062,"client":"192.168.2.21:59516","queuetimems":504606,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:46:37,269 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627697221","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:59516","queuetimems":508842,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:46:40,487 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627700439","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:59516","queuetimems":512060,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:46:40,490 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627700441","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:59516","queuetimems":511976,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:46:40,503 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627700452","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:59516","queuetimems":511948,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:47:10,094 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627970092","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":60001,"client":"192.168.2.21:59516","queuetimems":781613,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:47:10,510 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627730453","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300057,"client":"192.168.2.21:59516","queuetimems":541932,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:47:38,745 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: nonceCleaner average execution time: 93759 ns. 2020-11-17 15:47:38,746 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 173757 ns. 2020-11-17 15:47:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 43507 ns. 2020-11-17 15:48:47,827 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18776 ns. 2020-11-17 15:48:47,828 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 126702 ns. 2020-11-17 15:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44822019, hits=37952516, hitRatio=84.67%, , cachingAccesses=38006977, cachingHits=37945040, cachingHitsRatio=99.84%, evictions=5874, evicted=0, evictedPerRun=0.0 2020-11-17 15:50:38,674 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 78402 ns. 2020-11-17 15:50:38,746 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 45249 ns. 2020-11-17 15:50:57,801 INFO [regionserver/cvp21:16201.Chore.1] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. because e has an old edit so flush to free WALs after random delay 293200 ms 2020-11-17 15:51:10,262 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627670154","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 2 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":600107,"client":"192.168.2.21:59516","queuetimems":546216,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 15:51:10,545 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627970498","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:59516","queuetimems":781988,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:51:10,545 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627970492","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.21:59516","queuetimems":756434,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:51:23,316 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627983267","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:59516","queuetimems":794687,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:51:23,361 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627983311","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:59516","queuetimems":718749,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:51:23,361 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627983303","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300057,"client":"192.168.2.21:59516","queuetimems":769245,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:51:33,101 ERROR [MemStoreFlusher.0] regionserver.MemStoreFlusher: Cache flush failed for region aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync result after 300000 ms for txid=38185670, WAL system stuck? at org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:145) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:722) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:614) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:591) at org.apache.hadoop.hbase.regionserver.HRegion.doSyncOfUnflushedWALChanges(HRegion.java:2740) at org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2677) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2535) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2509) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2399) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:612) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:581) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:68) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:361) at java.lang.Thread.run(Thread.java:748) 2020-11-17 15:51:33,104 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 0469ce862509756add4a8cdb00bc465c 1/1 column families, dataSize=75.07 MB heapSize=91.69 MB 2020-11-17 15:51:37,340 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627997270","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":300070,"client":"192.168.2.21:59516","queuetimems":783212,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:51:38,002 INFO [regionserver/cvp21:16201.Chore.1] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. because e has an old edit so flush to free WALs after random delay 160130 ms 2020-11-17 15:51:40,536 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628000490","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:59516","queuetimems":736093,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:51:40,564 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628000504","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300060,"client":"192.168.2.21:59516","queuetimems":721804,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:52:10,155 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628030095","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300060,"client":"192.168.2.21:59516","queuetimems":765387,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:52:10,557 ERROR [MemStoreFlusher.1] regionserver.MemStoreFlusher: Cache flush failed for region aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync result after 300000 ms for txid=38185680, WAL system stuck? at org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:145) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:722) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:614) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:591) at org.apache.hadoop.hbase.regionserver.HRegion.doSyncOfUnflushedWALChanges(HRegion.java:2740) at org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2677) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2535) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2509) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2399) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:612) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:581) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:68) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:361) at java.lang.Thread.run(Thread.java:748) 2020-11-17 15:52:10,560 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627972489","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":358071,"client":"192.168.2.21:59516","queuetimems":783967,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:52:10,560 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627993050","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":337510,"client":"192.168.2.21:59516","queuetimems":714408,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:52:10,561 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 73aae5715542e1a5370f7540d2117abc 1/1 column families, dataSize=84.59 MB heapSize=125.97 MB 2020-11-17 15:52:10,560 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627992960","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":337600,"client":"192.168.2.21:59516","queuetimems":714335,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:52:10,560 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628030511","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:59516","queuetimems":726325,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:52:10,563 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605627992936","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":337627,"client":"192.168.2.21:59516","queuetimems":714337,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:52:10,566 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628000502","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":330064,"client":"192.168.2.21:59516","queuetimems":721860,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:53:10,564 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628330563","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":60001,"client":"192.168.2.21:59516","queuetimems":961943,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:53:10,564 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628330563","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":60001,"client":"192.168.2.21:59516","queuetimems":961943,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:53:38,567 INFO [regionserver/cvp21:16201.Chore.3] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. because e has an old edit so flush to free WALs after random delay 138873 ms 2020-11-17 15:53:38,711 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 42670 ns. 2020-11-17 15:53:38,749 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: nonceCleaner average execution time: 38883 ns. 2020-11-17 15:53:38,749 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 151198 ns. 2020-11-17 15:53:47,896 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionChecker average execution time: 17858 ns. 2020-11-17 15:53:47,896 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 507119 ns. 2020-11-17 15:54:10,571 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628390570","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":60001,"client":"192.168.2.21:59516","queuetimems":996264,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:54:27,779 INFO [regionserver/cvp21:16201.Chore.2] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. because e has an old edit so flush to free WALs after random delay 255050 ms 2020-11-17 15:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44822136, hits=37952633, hitRatio=84.67%, , cachingAccesses=38007094, cachingHits=37945157, cachingHitsRatio=99.84%, evictions=5904, evicted=0, evictedPerRun=0.0 2020-11-17 15:55:38,709 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 58911 ns. 2020-11-17 15:55:57,781 INFO [regionserver/cvp21:16201.Chore.3] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. because e has an old edit so flush to free WALs after random delay 159582 ms 2020-11-17 15:55:57,781 INFO [regionserver/cvp21:16201.Chore.3] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. because e has an old edit so flush to free WALs after random delay 90343 ms 2020-11-17 15:56:10,316 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628270262","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.21:59516","queuetimems":991763,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:56:10,597 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628270545","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:59516","queuetimems":992046,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:56:23,369 WARN [MemStoreFlusher.1] regionserver.DefaultMemStore: Snapshot called again without clearing previous. Doing nothing. Another ongoing flush or did we fail last attempt? 2020-11-17 15:56:23,416 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628283363","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.21:59516","queuetimems":1004777,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:56:23,418 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628283365","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.21:59516","queuetimems":928677,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:56:37,389 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628297342","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:59516","queuetimems":1018712,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:56:38,750 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 32883 ns. 2020-11-17 15:56:40,592 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628300540","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:59516","queuetimems":946023,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:56:40,616 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628300564","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:59516","queuetimems":945735,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:57:10,213 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628330157","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300056,"client":"192.168.2.21:59516","queuetimems":961538,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:57:10,607 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628330561","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:59516","queuetimems":961855,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:57:10,612 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628330563","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:59516","queuetimems":961943,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:57:10,615 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628330567","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:59516","queuetimems":961816,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:57:10,621 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628330565","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300056,"client":"192.168.2.21:59516","queuetimems":961845,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:58:10,619 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628390566","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:59516","queuetimems":1021804,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:58:58,115 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionChecker average execution time: 16190 ns. 2020-11-17 15:58:58,115 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 97430 ns. 2020-11-17 15:59:10,625 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628450573","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x90","processingtimems":300051,"client":"192.168.2.21:59516","queuetimems":1050158,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 15:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44822280, hits=37952777, hitRatio=84.67%, , cachingAccesses=38007238, cachingHits=37945301, cachingHitsRatio=99.84%, evictions=5934, evicted=0, evictedPerRun=0.0 2020-11-17 15:59:38,712 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: nonceCleaner average execution time: 54461 ns. 2020-11-17 15:59:38,715 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 42304 ns. 2020-11-17 15:59:38,715 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 190021 ns. 2020-11-17 16:00:38,734 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 679570 ns. 2020-11-17 16:01:10,378 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628570317","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300060,"client":"192.168.2.21:59516","queuetimems":1201571,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:01:10,671 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628270548","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 2 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":600123,"client":"192.168.2.21:59516","queuetimems":966361,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 16:01:10,671 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628570597","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300074,"client":"192.168.2.21:59516","queuetimems":1201835,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:01:23,415 ERROR [MemStoreFlusher.1] regionserver.MemStoreFlusher: Cache flush failed for region aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync result after 300000 ms for txid=38185734, WAL system stuck? at org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:145) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:722) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:614) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:591) at org.apache.hadoop.hbase.regionserver.HRegion.doSyncOfUnflushedWALChanges(HRegion.java:2740) at org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2677) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2535) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2509) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2399) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:612) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:581) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:68) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:361) at java.lang.Thread.run(Thread.java:748) 2020-11-17 16:01:23,424 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing fc93f6dcc564341dcc983bf30971f73d 1/1 column families, dataSize=36.79 MB heapSize=43.94 MB 2020-11-17 16:01:23,422 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628283318","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 2 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":600103,"client":"192.168.2.21:59516","queuetimems":1004818,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 16:01:23,467 ERROR [MemStoreFlusher.0] regionserver.MemStoreFlusher: Cache flush failed for region aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync result after 300000 ms for txid=38185736, WAL system stuck? at org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:145) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:722) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:614) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:591) at org.apache.hadoop.hbase.regionserver.HRegion.doSyncOfUnflushedWALChanges(HRegion.java:2740) at org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2677) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2535) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2509) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2399) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:612) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:581) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:68) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:361) at java.lang.Thread.run(Thread.java:748) 2020-11-17 16:01:23,467 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=40.74 MB heapSize=51.28 MB 2020-11-17 16:01:23,474 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628583422","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:59516","queuetimems":1124680,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:01:23,478 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628583419","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300059,"client":"192.168.2.21:59516","queuetimems":1214599,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:01:28,666 INFO [regionserver/cvp21:16201.Chore.3] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. because e has an old edit so flush to free WALs after random delay 29445 ms 2020-11-17 16:01:37,445 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628597395","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":300050,"client":"192.168.2.21:59516","queuetimems":1203089,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:01:40,677 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628600593","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300084,"client":"192.168.2.21:59516","queuetimems":1206286,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:01:40,678 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628600616","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300061,"client":"192.168.2.21:59516","queuetimems":1141789,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:02:10,264 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628630218","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.21:59516","queuetimems":1185579,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:02:10,679 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628630622","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300057,"client":"192.168.2.21:59516","queuetimems":1171750,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:02:10,679 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628630617","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300062,"client":"192.168.2.21:59516","queuetimems":1171751,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:02:10,681 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628630609","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300071,"client":"192.168.2.21:59516","queuetimems":1185800,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:02:10,685 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628630614","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300070,"client":"192.168.2.21:59516","queuetimems":1171773,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:02:38,746 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 44857 ns. 2020-11-17 16:02:40,680 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628900679","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":60001,"client":"192.168.2.21:53194","queuetimems":30296,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:03:10,687 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628930685","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":60002,"client":"192.168.2.21:53194","queuetimems":44117,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:04:08,277 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionChecker average execution time: 24274 ns. 2020-11-17 16:04:08,279 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 326772 ns. 2020-11-17 16:04:10,701 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628750627","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300074,"client":"192.168.2.21:59516","queuetimems":1215698,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44822527, hits=37953024, hitRatio=84.67%, , cachingAccesses=38007485, cachingHits=37945548, cachingHitsRatio=99.84%, evictions=5964, evicted=0, evictedPerRun=0.0 2020-11-17 16:05:10,704 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629050703","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":60000,"client":"192.168.2.21:53194","queuetimems":163823,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:05:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 39285 ns. 2020-11-17 16:05:38,710 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 201091 ns. 2020-11-17 16:05:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 71520 ns. 2020-11-17 16:06:10,427 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628870382","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x90","processingtimems":300045,"client":"192.168.2.21:53194","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:06:10,724 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628870673","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.22:58886","queuetimems":290,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:06:10,726 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605628870677","responsesize":"1667","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 2 action(s) and 1st row key\u003d\\x02\\x07\\x92\\xC4\\x09snapshots\\xC4\\x06config","processingtimems":300049,"client":"192.168.2.21:53194","queuetimems":295,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:06:23,474 ERROR [MemStoreFlusher.1] regionserver.MemStoreFlusher: Cache flush failed for region aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync result after 300000 ms for txid=38185768, WAL system stuck? at org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:145) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:722) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:614) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:591) at org.apache.hadoop.hbase.regionserver.HRegion.doSyncOfUnflushedWALChanges(HRegion.java:2740) at org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2677) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2535) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2509) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2399) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:612) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:581) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:68) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:361) at java.lang.Thread.run(Thread.java:748) 2020-11-17 16:06:23,476 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=774.72 KB heapSize=1.01 MB 2020-11-17 16:06:23,483 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":22,"starttimems":"1605628883426","responsesize":"1838","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 23 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x0AztpService\\xC4\\x06config\\x \u003cTRUNCATED\u003e","processingtimems":300056,"client":"192.168.2.21:53194","queuetimems":13042,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:06:23,522 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628883476","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.22:58886","queuetimems":13094,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:06:23,530 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628883478","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:53194","queuetimems":12801,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:06:28,448 INFO [regionserver/cvp21:16201.Chore.3] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. because e has an old edit so flush to free WALs after random delay 231825 ms 2020-11-17 16:06:37,494 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628897447","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x95\\xC4\\x0AztpService\\xC4\\x06device\\xC \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:53194","queuetimems":26770,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:06:38,658 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 58418 ns. 2020-11-17 16:06:40,724 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628900678","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.22:58886","queuetimems":29999,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:07:10,313 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628930266","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x94\\xC4\\x09BugAlerts\\xC4\\x06update\\xC4 \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.22:58886","queuetimems":43731,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:07:10,733 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628930685","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:53194","queuetimems":60009,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:07:10,735 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":3,"starttimems":"1605628930685","responsesize":"3337","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 5 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x0AztpService\\xC4\\x06config\\xC \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:53194","queuetimems":60302,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 16:07:10,738 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628930686","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:53194","queuetimems":60009,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:07:23,533 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629183532","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":60001,"client":"192.168.2.21:53194","queuetimems":282732,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:07:40,736 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628960682","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.21:53194","queuetimems":73945,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:08:10,740 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628990687","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.21:53194","queuetimems":90018,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:08:10,740 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605628690621","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 2 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":600119,"client":"192.168.2.21:59516","queuetimems":1206193,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 16:08:38,747 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 71990 ns. 2020-11-17 16:09:08,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14968 ns. 2020-11-17 16:09:08,714 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 80703 ns. 2020-11-17 16:09:10,745 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629290744","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":60001,"client":"192.168.2.21:53194","queuetimems":299824,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44822562, hits=37953059, hitRatio=84.67%, , cachingAccesses=38007520, cachingHits=37945583, cachingHitsRatio=99.84%, evictions=5994, evicted=0, evictedPerRun=0.0 2020-11-17 16:10:10,759 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629110706","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:53194","queuetimems":210037,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:11:10,768 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629170724","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300043,"client":"192.168.2.21:53194","queuetimems":269967,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:11:10,771 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629170728","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300043,"client":"192.168.2.21:53194","queuetimems":270059,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:11:23,533 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629183485","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:53194","queuetimems":282716,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:11:23,533 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629170427","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":313106,"client":"192.168.2.21:53194","queuetimems":269758,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:11:23,536 ERROR [MemStoreFlusher.0] regionserver.MemStoreFlusher: Cache flush failed for region aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync result after 300000 ms for txid=38185800, WAL system stuck? at org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:145) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:722) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:614) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:591) at org.apache.hadoop.hbase.regionserver.HRegion.doSyncOfUnflushedWALChanges(HRegion.java:2740) at org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2677) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2535) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2509) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2399) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:612) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:581) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:68) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:361) at java.lang.Thread.run(Thread.java:748) 2020-11-17 16:11:23,537 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 82421ce8c5c8b48c573baee83caa8305 1/1 column families, dataSize=2.92 MB heapSize=3.54 MB 2020-11-17 16:11:23,585 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629183523","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300061,"client":"192.168.2.21:53194","queuetimems":282727,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:11:37,544 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629197495","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:53194","queuetimems":296683,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:11:38,662 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 67484 ns. 2020-11-17 16:11:38,704 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 194261 ns. 2020-11-17 16:11:38,746 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: nonceCleaner average execution time: 85489 ns. 2020-11-17 16:11:38,747 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 59803 ns. 2020-11-17 16:11:40,771 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629200725","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:53194","queuetimems":299913,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:12:10,356 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629230314","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300041,"client":"192.168.2.21:53194","queuetimems":329443,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:12:10,784 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629230734","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":300050,"client":"192.168.2.21:53194","queuetimems":304378,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:12:10,791 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629230740","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:53194","queuetimems":239951,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:12:10,792 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629230740","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:53194","queuetimems":254051,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:12:23,597 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629243541","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300055,"client":"192.168.2.21:53194","queuetimems":317185,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:12:40,788 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629260738","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:53194","queuetimems":269948,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:13:10,794 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629290744","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:53194","queuetimems":364388,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:14:10,801 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629350745","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300056,"client":"192.168.2.21:53194","queuetimems":359753,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:14:17,982 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionChecker average execution time: 15997 ns. 2020-11-17 16:14:17,982 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 49189 ns. 2020-11-17 16:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44822650, hits=37953147, hitRatio=84.67%, , cachingAccesses=38007608, cachingHits=37945671, cachingHitsRatio=99.84%, evictions=6024, evicted=0, evictedPerRun=0.0 2020-11-17 16:14:38,748 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 60881 ns. 2020-11-17 16:15:10,820 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629410763","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300056,"client":"192.168.2.21:53194","queuetimems":394287,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:16:10,822 ERROR [MemStoreFlusher.1] regionserver.MemStoreFlusher: Cache flush failed for region aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync result after 300000 ms for txid=38185830, WAL system stuck? at org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:145) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:722) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:614) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:591) at org.apache.hadoop.hbase.regionserver.HRegion.doSyncOfUnflushedWALChanges(HRegion.java:2740) at org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2677) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2535) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2509) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2399) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:612) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:581) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:68) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:361) at java.lang.Thread.run(Thread.java:748) 2020-11-17 16:16:10,823 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629470771","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:53194","queuetimems":493912,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:16:10,825 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=41.07 MB heapSize=49.45 MB 2020-11-17 16:16:10,827 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629470773","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":300053,"client":"192.168.2.21:53194","queuetimems":454296,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:16:17,778 INFO [regionserver/cvp21:16201.Chore.1] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. because e has an old edit so flush to free WALs after random delay 34093 ms 2020-11-17 16:16:23,644 ERROR [MemStoreFlusher.0] regionserver.MemStoreFlusher: Cache flush failed for region aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync result after 300000 ms for txid=38185840, WAL system stuck? at org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:145) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:722) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:614) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:591) at org.apache.hadoop.hbase.regionserver.HRegion.doSyncOfUnflushedWALChanges(HRegion.java:2740) at org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2677) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2535) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2509) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2399) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:612) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:581) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:68) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:361) at java.lang.Thread.run(Thread.java:748) 2020-11-17 16:16:23,646 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629483535","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300111,"client":"192.168.2.21:53194","queuetimems":492746,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:16:23,646 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629483594","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:53194","queuetimems":402684,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:16:23,647 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 06f7ca9534710e37d90d9f8345864259 1/1 column families, dataSize=40.74 MB heapSize=51.28 MB 2020-11-17 16:16:23,646 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629483535","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300109,"client":"192.168.2.21:53194","queuetimems":506535,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:16:37,647 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629497545","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300102,"client":"192.168.2.21:53194","queuetimems":506755,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:16:38,726 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 78451 ns. 2020-11-17 16:16:40,831 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629500771","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300059,"client":"192.168.2.21:53194","queuetimems":509893,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:17:10,421 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629530360","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300060,"client":"192.168.2.22:44686","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:17:10,841 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629530793","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x90","processingtimems":300048,"client":"192.168.2.21:43358","queuetimems":3,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:17:10,845 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629530792","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.22:44686","queuetimems":4,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:17:23,652 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629543598","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.21:43358","queuetimems":13237,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:17:38,710 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 72972 ns. 2020-11-17 16:17:38,746 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: nonceCleaner average execution time: 107034 ns. 2020-11-17 16:17:38,748 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 586821 ns. 2020-11-17 16:17:40,833 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629800832","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":60001,"client":"192.168.2.21:50310","queuetimems":29998,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:17:40,843 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629560790","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.21:43358","queuetimems":30000,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:18:10,844 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629830843","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":60000,"client":"192.168.2.21:50310","queuetimems":60009,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:18:10,845 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629590796","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:43358","queuetimems":60434,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:19:10,847 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629890846","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":60001,"client":"192.168.2.21:52730","queuetimems":30008,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:19:10,847 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629890846","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":60001,"client":"192.168.2.21:52730","queuetimems":30008,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:19:10,853 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629650805","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:43358","queuetimems":119293,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:19:27,827 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 42760 ns. 2020-11-17 16:19:27,827 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 18477 ns. 2020-11-17 16:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44822711, hits=37953208, hitRatio=84.67%, , cachingAccesses=38007669, cachingHits=37945732, cachingHitsRatio=99.84%, evictions=6054, evicted=0, evictedPerRun=0.0 2020-11-17 16:19:38,785 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 30656 ns. 2020-11-17 16:20:10,866 WARN [MemStoreFlusher.0] regionserver.DefaultMemStore: Snapshot called again without clearing previous. Doing nothing. Another ongoing flush or did we fail last attempt? 2020-11-17 16:20:10,868 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629710823","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300043,"client":"192.168.2.21:48242","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:21:10,877 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629770830","responsesize":"3313","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 2 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.22:50184","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 16:21:10,886 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629770830","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300056,"client":"192.168.2.22:50184","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:22:10,468 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629830421","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:50310","queuetimems":59586,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:22:10,886 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629530788","responsesize":"11586","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 7 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":600097,"client":"192.168.2.22:44686","queuetimems":428,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":7} 2020-11-17 16:22:10,892 ERROR [MemStoreFlusher.1] regionserver.MemStoreFlusher: Cache flush failed for region aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync result after 300000 ms for txid=38185874, WAL system stuck? at org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:145) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:722) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:614) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:591) at org.apache.hadoop.hbase.regionserver.HRegion.doSyncOfUnflushedWALChanges(HRegion.java:2740) at org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2677) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2535) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2509) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2399) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:612) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:581) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:68) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:361) at java.lang.Thread.run(Thread.java:748) 2020-11-17 16:22:10,895 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 5e5f05205d7e40b0141cd977dd009ba1 1/1 column families, dataSize=774.72 KB heapSize=1.01 MB 2020-11-17 16:22:10,895 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605629783647","responsesize":"1667","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 2 action(s) and 1st row key\u003d\\x02\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":347244,"client":"192.168.2.22:50184","queuetimems":12816,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:22:10,895 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629783647","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":347248,"client":"192.168.2.22:50184","queuetimems":12816,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:22:10,900 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629830847","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.21:50310","queuetimems":60012,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:22:10,903 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629783650","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":347253,"client":"192.168.2.21:50310","queuetimems":12816,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:22:10,904 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605629797648","responsesize":"1667","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 2 action(s) and 1st row key\u003d\\x02\\x07\\x90","processingtimems":333256,"client":"192.168.2.21:50310","queuetimems":26814,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:22:23,697 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629843654","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300043,"client":"192.168.2.21:50310","queuetimems":72819,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:22:38,658 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 84457 ns. 2020-11-17 16:22:40,890 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605629860845","responsesize":"1667","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 2 action(s) and 1st row key\u003d\\x02\\x07\\x92\\xC4\\x09snapshots\\xC4\\x06config","processingtimems":300044,"client":"192.168.2.22:51780","queuetimems":8,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:22:40,892 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629860837","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.22:51780","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:23:10,898 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630130897","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":60001,"client":"192.168.2.21:60348","queuetimems":423,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:23:10,902 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630130902","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":60000,"client":"192.168.2.21:60348","queuetimems":428,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:23:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 44953 ns. 2020-11-17 16:23:38,712 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: nonceCleaner average execution time: 172279 ns. 2020-11-17 16:23:38,712 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 929740 ns. 2020-11-17 16:24:10,901 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629950850","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x95\\xC4\\x0AztpService\\xC4\\x06device\\xC \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:52730","queuetimems":90012,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:24:10,907 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629950850","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300057,"client":"192.168.2.21:52730","queuetimems":90012,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:24:10,912 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605629950856","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300056,"client":"192.168.2.21:52730","queuetimems":90017,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:24:10,917 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948954 2020-11-17 16:24:28,473 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 14524 ns. 2020-11-17 16:24:28,473 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 58049 ns. 2020-11-17 16:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44823268, hits=37953765, hitRatio=84.67%, , cachingAccesses=38008226, cachingHits=37946289, cachingHitsRatio=99.84%, evictions=6084, evicted=0, evictedPerRun=0.0 2020-11-17 16:25:10,921 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630010868","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.21:52730","queuetimems":150029,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:25:10,922 ERROR [MemStoreFlusher.0] regionserver.MemStoreFlusher: Cache flush failed for region aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync result after 300000 ms for txid=38185898, WAL system stuck? at org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:145) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:722) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:614) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:591) at org.apache.hadoop.hbase.regionserver.HRegion.doSyncOfUnflushedWALChanges(HRegion.java:2740) at org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2677) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2535) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2509) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2399) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:612) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:581) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:68) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:361) at java.lang.Thread.run(Thread.java:748) 2020-11-17 16:25:10,923 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing e5dc5c620be19b02666a19c280ff582e 1/1 column families, dataSize=41.07 MB heapSize=49.45 MB 2020-11-17 16:25:38,746 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 50702 ns. 2020-11-17 16:26:10,925 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630310925","responsesize":"1380","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":60000,"client":"192.168.2.22:33384","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:26:10,931 WARN [MemStoreFlusher.1] regionserver.DefaultMemStore: Snapshot called again without clearing previous. Doing nothing. Another ongoing flush or did we fail last attempt? 2020-11-17 16:26:10,931 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630070882","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.22:56432","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:26:10,936 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":23,"starttimems":"1605630070890","responsesize":"1843","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 24 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x0AztpService\\xC4\\x06config\\x \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:57952","queuetimems":7,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:26:10,939 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948958 2020-11-17 16:26:10,941 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948957 2020-11-17 16:27:10,522 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630130473","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.22:57724","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:27:10,945 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630130897","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x0BeventAlerts\\xC4\\x14alertman \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.23:41418","queuetimems":424,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:27:10,958 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630130906","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":300052,"client":"192.168.2.21:60348","queuetimems":430,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:27:10,966 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630130904","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x99\\xC4\\x07Devices\\xC4\\x0BSSN19501010\\ \u003cTRUNCATED\u003e","processingtimems":300061,"client":"192.168.2.23:41418","queuetimems":429,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:27:23,747 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630143699","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:60348","queuetimems":13223,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:27:38,747 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 67850 ns. 2020-11-17 16:27:40,943 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630160898","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.21:60348","queuetimems":30417,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:27:40,947 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630160893","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x90","processingtimems":300054,"client":"192.168.2.21:60348","queuetimems":30417,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:28:10,950 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630190900","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:60348","queuetimems":60424,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:28:10,960 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630190910","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:60348","queuetimems":60428,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:29:10,959 WARN [MemStoreFlusher.0] regionserver.DefaultMemStore: Snapshot called again without clearing previous. Doing nothing. Another ongoing flush or did we fail last attempt? 2020-11-17 16:29:10,961 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630250905","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.22:60326","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44823531, hits=37954028, hitRatio=84.67%, , cachingAccesses=38008489, cachingHits=37946552, cachingHitsRatio=99.84%, evictions=6114, evicted=0, evictedPerRun=0.0 2020-11-17 16:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 16:29:38,729 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 51619 ns. 2020-11-17 16:29:38,729 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 16794 ns. 2020-11-17 16:29:38,730 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 537952 ns. 2020-11-17 16:29:38,730 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 46502 ns. 2020-11-17 16:29:38,731 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 203390 ns. 2020-11-17 16:31:10,973 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605630250918","responsesize":"17086","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 12 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ16292009 \u003cTRUNCATED\u003e","processingtimems":420054,"client":"192.168.2.23:44286","queuetimems":12,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":11} 2020-11-17 16:31:10,992 ERROR [MemStoreFlusher.1] regionserver.MemStoreFlusher: Cache flush failed for region aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync result after 300000 ms for txid=38185934, WAL system stuck? at org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:145) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:722) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:614) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:591) at org.apache.hadoop.hbase.regionserver.HRegion.doSyncOfUnflushedWALChanges(HRegion.java:2740) at org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2677) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2535) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2509) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2399) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:612) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:581) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:68) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:361) at java.lang.Thread.run(Thread.java:748) 2020-11-17 16:31:10,993 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630370945","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:37088","queuetimems":60018,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:31:10,996 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":8,"starttimems":"1605630370941","responsesize":"1723","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 9 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300055,"client":"192.168.2.21:37088","queuetimems":60014,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:31:38,747 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 42481 ns. 2020-11-17 16:32:10,581 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630430527","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.22:36048","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:32:10,994 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630430945","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.22:36048","queuetimems":418,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:32:10,994 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948953 2020-11-17 16:32:11,010 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630430959","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x94\\xC4\\x09BugAlerts\\xC4\\x06update\\xC4 \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.22:36048","queuetimems":432,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:32:11,012 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948963 2020-11-17 16:32:11,020 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630430966","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.22:36048","queuetimems":439,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:32:11,021 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948960 2020-11-17 16:32:11,022 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948955 2020-11-17 16:32:23,801 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: Can not complete this request in time, drop it: callId: 9 service: ClientService methodName: Mutate size: 372 connection: 192.168.2.23:49912 deadline: 9223372036854775807 param: region= aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., row=\x0B\x07\x93\xC4\x09snapshots\xC4\x06config\xC4\x0DrequestStatus connection: 192.168.2.23:49912 2020-11-17 16:32:40,996 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630460945","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.22:36048","queuetimems":30417,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:32:41,004 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630460949","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.22:36048","queuetimems":30423,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:33:38,658 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 367206 ns. 2020-11-17 16:34:11,004 ERROR [MemStoreFlusher.0] regionserver.MemStoreFlusher: Cache flush failed for region aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync result after 300000 ms for txid=38185960, WAL system stuck? at org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:145) at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:722) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:614) at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:591) at org.apache.hadoop.hbase.regionserver.HRegion.doSyncOfUnflushedWALChanges(HRegion.java:2740) at org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2677) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2535) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2509) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2399) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:612) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:581) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:68) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:361) at java.lang.Thread.run(Thread.java:748) 2020-11-17 16:34:11,031 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630550967","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x94\\xC4\\x0Acompliance\\xC4\\x0BJPE144830 \u003cTRUNCATED\u003e","processingtimems":300064,"client":"192.168.2.23:52738","queuetimems":2,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44823792, hits=37954289, hitRatio=84.67%, , cachingAccesses=38008750, cachingHits=37946813, cachingHitsRatio=99.84%, evictions=6144, evicted=0, evictedPerRun=0.0 2020-11-17 16:34:47,843 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 23366 ns. 2020-11-17 16:34:47,843 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 38683 ns. 2020-11-17 16:35:38,723 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 247467 ns. 2020-11-17 16:35:38,723 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 62475 ns. 2020-11-17 16:35:38,723 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: nonceCleaner average execution time: 230314 ns. 2020-11-17 16:36:11,048 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630670980","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300067,"client":"192.168.2.22:41098","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:36:11,048 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605630370926","responsesize":"8018","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 5 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":600122,"client":"192.168.2.22:33384","queuetimems":60001,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":4} 2020-11-17 16:36:11,054 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948965 2020-11-17 16:36:11,057 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630670999","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300057,"client":"192.168.2.22:41098","queuetimems":19,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:37:10,630 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630730583","responsesize":"6621","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 4 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:46244","queuetimems":59601,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":4} 2020-11-17 16:37:11,044 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605630130895","responsesize":"8022","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 5 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":900149,"client":"192.168.2.22:57724","queuetimems":422,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":4} 2020-11-17 16:37:11,048 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948956 2020-11-17 16:37:11,055 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1605630731003","responsesize":"1677","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 3 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x0AztpService\\xC4\\x06config\\xC \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:46244","queuetimems":60002,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:37:11,060 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948968 2020-11-17 16:37:11,061 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948959 2020-11-17 16:37:11,061 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948962 2020-11-17 16:37:11,061 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948961 2020-11-17 16:37:11,064 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630730995","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300068,"client":"192.168.2.21:46244","queuetimems":60011,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:37:11,067 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":8,"starttimems":"1605630731012","responsesize":"3377","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 10 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x0AztpService\\xC4\\x06config\\x \u003cTRUNCATED\u003e","processingtimems":300055,"client":"192.168.2.21:46244","queuetimems":60009,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 16:37:23,849 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630743802","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x94\\xC4\\x09BugAlerts\\xC4\\x06update\\xC4 \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.22:41098","queuetimems":54846,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:37:38,773 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 33395 ns. 2020-11-17 16:37:41,045 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630760997","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:46244","queuetimems":72010,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:37:41,051 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630761004","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:46244","queuetimems":90024,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:38:11,085 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630490961","responsesize":"129022","method":"Multi","param":"region\u003d aeris_v2,\\x06\\xCD\\x04\\x16\\x97\\xC4\\x05Smash\\xC4\\x07routing\\xC4\\x03pim\\xC4\\x0Asparsemode\\xC4\\x06status\\xC4\\x07default\\xC4\\x09routeIntf,160496158 \u003cTRUNCATED\u003e","processingtimems":600123,"client":"192.168.2.23:49732","queuetimems":60401,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":78} 2020-11-17 16:39:11,087 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630851035","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x95\\xC4\\x0AztpService\\xC4\\x06device\\xC \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:51192","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:39:11,090 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605630250912","responsesize":"11330","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 7 action(s) and 1st row key\u003d\\x07\\x06\\x94\\xC4\\x09BugAlerts\\xC4\\x06update\\xC4 \u003cTRUNCATED\u003e","processingtimems":900178,"client":"192.168.2.22:60326","queuetimems":7,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":6} 2020-11-17 16:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44824131, hits=37954628, hitRatio=84.67%, , cachingAccesses=38009089, cachingHits=37947152, cachingHitsRatio=99.84%, evictions=6174, evicted=0, evictedPerRun=0.0 2020-11-17 16:39:38,796 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 117905 ns. 2020-11-17 16:39:47,894 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 16732 ns. 2020-11-17 16:39:47,894 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 54462 ns. 2020-11-17 16:41:11,110 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630971055","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.22:47820","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:41:11,110 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630971056","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.21:54594","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:41:11,110 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630971060","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ16421238\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.23:37650","queuetimems":3,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:41:11,115 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948971 2020-11-17 16:41:38,718 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 1565207 ns. 2020-11-17 16:41:38,721 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 2475363 ns. 2020-11-17 16:41:38,718 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 215745 ns. 2020-11-17 16:42:10,674 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631030631","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x90","processingtimems":300043,"client":"192.168.2.21:54594","queuetimems":59579,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:42:11,097 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631031053","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300043,"client":"192.168.2.21:54594","queuetimems":57408,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:42:11,118 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631031063","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300055,"client":"192.168.2.21:54594","queuetimems":31936,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:42:11,120 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631031069","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x95\\xC4\\x0AztpService\\xC4\\x06device\\xC \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:54594","queuetimems":437,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:42:11,121 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631031068","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.21:54594","queuetimems":436,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:42:23,911 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631043850","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300061,"client":"192.168.2.21:54594","queuetimems":13217,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:42:41,098 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631061046","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:54594","queuetimems":30412,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:42:41,101 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631061052","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:54594","queuetimems":61924,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:43:11,139 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631091090","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:54594","queuetimems":60456,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:43:38,748 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 49715 ns. 2020-11-17 16:44:11,143 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631151094","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:59544","queuetimems":2,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:44:11,147 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631151094","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:59544","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44824253, hits=37954750, hitRatio=84.67%, , cachingAccesses=38009211, cachingHits=37947274, cachingHitsRatio=99.84%, evictions=6204, evicted=0, evictedPerRun=0.0 2020-11-17 16:44:57,796 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionChecker average execution time: 14483 ns. 2020-11-17 16:44:57,796 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 38226 ns. 2020-11-17 16:45:38,684 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 55596 ns. 2020-11-17 16:46:11,163 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631271115","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.22:54310","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:46:11,171 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: Can not complete this request in time, drop it: callId: 9 service: ClientService methodName: Mutate size: 372 connection: 192.168.2.23:46350 deadline: 9223372036854775807 param: region= aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., row=\x0B\x07\x93\xC4\x09snapshots\xC4\x06config\xC4\x0DrequestStatus connection: 192.168.2.23:46350 2020-11-17 16:47:23,971 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605631343915","responsesize":"29786","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 19 action(s) and 1st row key\u003d\\x02\\x07\\x92\\xC4\\x09snapshots\\xC4\\x06config","processingtimems":300056,"client":"192.168.2.23:47206","queuetimems":13235,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":18} 2020-11-17 16:47:38,750 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 1552750 ns. 2020-11-17 16:47:38,750 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 2491763 ns. 2020-11-17 16:47:38,750 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: nonceCleaner average execution time: 225021 ns. 2020-11-17 16:47:41,159 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631361103","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300055,"client":"192.168.2.22:55512","queuetimems":30424,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:47:41,159 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948977 2020-11-17 16:47:41,159 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948973 2020-11-17 16:47:41,161 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948972 2020-11-17 16:47:41,162 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948970 2020-11-17 16:47:41,172 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631361123","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.22:55512","queuetimems":30023,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:47:41,174 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631361102","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280689\\ \u003cTRUNCATED\u003e","processingtimems":300072,"client":"192.168.2.23:47206","queuetimems":30424,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:47:41,176 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631361126","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x94\\xC4\\x09BugAlerts\\xC4\\x06update\\xC4 \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.22:55512","queuetimems":30024,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:48:11,166 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630490952","responsesize":"133991","method":"Multi","param":"region\u003d aeris_v2,\\x06\\xCD\\x04\\x16\\x97\\xC4\\x05Smash\\xC4\\x07routing\\xC4\\x03pim\\xC4\\x0Asparsemode\\xC4\\x06status\\xC4\\x07default\\xC4\\x09routeIntf,160496158 \u003cTRUNCATED\u003e","processingtimems":1200213,"client":"192.168.2.21:40136","queuetimems":60393,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":81} 2020-11-17 16:48:23,977 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948975 2020-11-17 16:48:23,980 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948974 2020-11-17 16:48:23,981 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948976 2020-11-17 16:48:38,751 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 56436 ns. 2020-11-17 16:49:11,187 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631451147","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300039,"client":"192.168.2.21:36912","queuetimems":120467,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:49:11,203 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631451145","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x95\\xC4\\x0AztpService\\xC4\\x06device\\xC \u003cTRUNCATED\u003e","processingtimems":300058,"client":"192.168.2.21:36912","queuetimems":120042,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44824353, hits=37954850, hitRatio=84.67%, , cachingAccesses=38009311, cachingHits=37947374, cachingHitsRatio=99.84%, evictions=6234, evicted=0, evictedPerRun=0.0 2020-11-17 16:49:57,820 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 17841 ns. 2020-11-17 16:49:57,821 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 35737 ns. 2020-11-17 16:51:11,207 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631271119","responsesize":"13240","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 8 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":600088,"client":"192.168.2.22:54310","queuetimems":4,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":8} 2020-11-17 16:51:11,214 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: Can not complete this request in time, drop it: callId: 12 service: ClientService methodName: Mutate size: 372 connection: 192.168.2.23:54332 deadline: 9223372036854775807 param: region= aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., row=\x0B\x07\x93\xC4\x09snapshots\xC4\x06config\xC4\x0DrequestStatus connection: 192.168.2.23:54332 2020-11-17 16:51:11,243 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631571171","responsesize":"3313","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 2 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300072,"client":"192.168.2.22:33092","queuetimems":3,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 16:51:38,661 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 54877 ns. 2020-11-17 16:52:40,778 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631360682","responsesize":"24818","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 15 action(s) and 1st row key\u003d\\x0B\\xCD\\x04\\x1D\\x95\\xC4\\x05Smash\\xC4\\x07routi \u003cTRUNCATED\u003e","processingtimems":600095,"client":"192.168.2.23:47214","queuetimems":29973,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":15} 2020-11-17 16:52:41,221 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631661174","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BHNN20205024\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.23:54332","queuetimems":70972,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:52:41,222 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":6,"starttimems":"1605631661177","responsesize":"1709","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 7 action(s) and 1st row key\u003d\\x0B\\x02\\x93\\xC4\\x04oidc\\xC4\\x08sessions\\xC4i74 \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.23:54332","queuetimems":90009,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:53:11,211 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631691168","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300043,"client":"192.168.2.21:43842","queuetimems":100671,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:53:11,224 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631691176","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:43842","queuetimems":120006,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:53:24,034 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631703983","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:43842","queuetimems":113790,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:53:38,704 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 346108 ns. 2020-11-17 16:53:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 52541 ns. 2020-11-17 16:53:38,711 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: nonceCleaner average execution time: 425234 ns. 2020-11-17 16:54:11,237 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631751187","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:43842","queuetimems":146901,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:54:11,271 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631751203","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300068,"client":"192.168.2.21:43842","queuetimems":160849,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44824583, hits=37955080, hitRatio=84.67%, , cachingAccesses=38009541, cachingHits=37947604, cachingHitsRatio=99.84%, evictions=6264, evicted=0, evictedPerRun=0.0 2020-11-17 16:54:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 42775 ns. 2020-11-17 16:54:57,860 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionChecker average execution time: 16602 ns. 2020-11-17 16:54:57,860 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 38370 ns. 2020-11-17 16:56:11,269 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631871214","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.21:43842","queuetimems":266725,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:56:11,269 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631871208","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300060,"client":"192.168.2.21:43842","queuetimems":266785,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:56:11,288 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631871245","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300043,"client":"192.168.2.21:43842","queuetimems":241273,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:57:11,266 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631331122","responsesize":"33092","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 20 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ16292009 \u003cTRUNCATED\u003e","processingtimems":900143,"client":"192.168.2.23:47206","queuetimems":443,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":20} 2020-11-17 16:57:38,658 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 70586 ns. 2020-11-17 16:57:40,830 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631960780","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:43842","queuetimems":356407,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:57:41,264 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631961221","responsesize":"3313","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 2 action(s) and 1st row key\u003d\\x04\\x07\\x95\\xC4\\x0AztpService\\xC4\\x06device\\xC \u003cTRUNCATED\u003e","processingtimems":300042,"client":"192.168.2.21:43842","queuetimems":317243,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 16:57:41,276 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631961225","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":300050,"client":"192.168.2.21:43842","queuetimems":331253,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:58:11,257 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631991213","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.21:43842","queuetimems":347235,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:58:11,276 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631991229","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:43842","queuetimems":347247,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:58:24,095 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632004037","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300057,"client":"192.168.2.21:43842","queuetimems":323724,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:59:11,293 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632051237","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x90","processingtimems":300055,"client":"192.168.2.21:43842","queuetimems":407258,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:59:11,318 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632051273","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.21:43842","queuetimems":356867,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 16:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44824690, hits=37955187, hitRatio=84.67%, , cachingAccesses=38009648, cachingHits=37947711, cachingHitsRatio=99.84%, evictions=6294, evicted=0, evictedPerRun=0.0 2020-11-17 16:59:38,710 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 46769 ns. 2020-11-17 16:59:38,712 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 580866 ns. 2020-11-17 16:59:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 447698 ns. 2020-11-17 16:59:38,748 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 41914 ns. 2020-11-17 17:00:07,779 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 50902 ns. 2020-11-17 17:00:07,779 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 29816 ns. 2020-11-17 17:01:11,316 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632171269","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:43842","queuetimems":527290,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:01:11,323 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632171273","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:43842","queuetimems":527292,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:01:11,342 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632171290","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:43842","queuetimems":527308,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:02:11,315 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632231270","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.21:43842","queuetimems":511178,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:02:41,314 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":26,"starttimems":"1605631661164","responsesize":"8489","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 31 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4 \u003cTRUNCATED\u003e","processingtimems":900150,"client":"192.168.2.21:43842","queuetimems":89993,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":5} 2020-11-17 17:02:41,316 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632261264","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:43842","queuetimems":617282,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:02:41,321 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605631361100","responsesize":"271311","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 164 action(s) and 1st row key\u003d\\x0B\\x19\\x96\\xC4\\x05Sysdb\\xC4\\x0Benvironment\\ \u003cTRUNCATED\u003e","processingtimems":1200220,"client":"192.168.2.22:55526","queuetimems":30390,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":164} 2020-11-17 17:02:41,325 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632261279","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.21:43842","queuetimems":490845,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:03:11,305 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632291259","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:43842","queuetimems":647276,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:03:11,324 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632291277","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:43842","queuetimems":520680,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:03:24,151 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632304097","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.21:43842","queuetimems":533359,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:03:38,658 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 73964 ns. 2020-11-17 17:04:11,347 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632351298","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:43842","queuetimems":670822,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:04:11,366 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632351321","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.21:43842","queuetimems":566795,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44824706, hits=37955203, hitRatio=84.67%, , cachingAccesses=38009664, cachingHits=37947727, cachingHitsRatio=99.84%, evictions=6324, evicted=0, evictedPerRun=0.0 2020-11-17 17:05:07,779 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionChecker average execution time: 20002 ns. 2020-11-17 17:05:07,779 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 52507 ns. 2020-11-17 17:05:38,713 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 43101 ns. 2020-11-17 17:05:38,713 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: nonceCleaner average execution time: 462123 ns. 2020-11-17 17:05:38,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 557333 ns. 2020-11-17 17:05:38,749 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 51694 ns. 2020-11-17 17:06:11,365 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632471317","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:43842","queuetimems":790700,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:06:11,370 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632471323","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:43842","queuetimems":776918,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:06:11,392 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632471343","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:43842","queuetimems":776938,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:07:11,360 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632531317","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300042,"client":"192.168.2.21:43842","queuetimems":746791,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:07:40,940 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632260832","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 2 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":600107,"client":"192.168.2.21:43842","queuetimems":540739,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 17:07:41,363 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632561317","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:43842","queuetimems":866911,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:07:41,364 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632561318","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.21:43842","queuetimems":866824,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:07:41,370 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632561326","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.21:43842","queuetimems":776672,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:07:41,376 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632561322","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.21:43842","queuetimems":776707,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:08:11,357 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632591307","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:43842","queuetimems":896794,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:08:11,371 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632591325","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:43842","queuetimems":806660,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:08:11,387 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":7,"starttimems":"1605631391141","responsesize":"113986","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 70 action(s) and 1st row key\u003d\\x04\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BJPE19362489 \u003cTRUNCATED\u003e","processingtimems":1500246,"client":"192.168.2.23:47206","queuetimems":60461,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":63} 2020-11-17 17:08:24,205 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632604153","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:43842","queuetimems":819422,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:08:38,942 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 37722 ns. 2020-11-17 17:09:11,397 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632651349","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:43842","queuetimems":956816,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:09:11,419 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632651369","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:43842","queuetimems":790511,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44824768, hits=37955265, hitRatio=84.67%, , cachingAccesses=38009726, cachingHits=37947789, cachingHitsRatio=99.84%, evictions=6354, evicted=0, evictedPerRun=0.0 2020-11-17 17:10:08,145 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 27561 ns. 2020-11-17 17:10:08,146 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 31255 ns. 2020-11-17 17:11:11,410 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632771365","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.21:43842","queuetimems":1076821,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:11:11,418 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632771370","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:43842","queuetimems":1076821,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:11:11,438 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632771393","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.21:43842","queuetimems":1076844,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:11:38,715 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 46260 ns. 2020-11-17 17:11:38,717 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 242815 ns. 2020-11-17 17:11:38,719 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 513219 ns. 2020-11-17 17:11:38,746 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 31702 ns. 2020-11-17 17:12:11,412 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632831363","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:43842","queuetimems":956717,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:12:11,465 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605630731026","responsesize":"1026007","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 620 action(s) and 1st row key\u003d\\x04\\xCD\\x04\\x03\\x98\\xC4\\x05Sysdb\\xC4\\x0Benvi \u003cTRUNCATED\u003e","processingtimems":2400439,"client":"192.168.2.22:41108","queuetimems":60011,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":620} 2020-11-17 17:12:41,006 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632860942","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300064,"client":"192.168.2.21:43842","queuetimems":986188,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:12:41,412 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632861366","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:43842","queuetimems":1166756,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:12:41,417 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632861370","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:43842","queuetimems":1076843,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:12:41,418 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632861371","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:43842","queuetimems":986580,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:12:41,423 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632861377","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:43842","queuetimems":961042,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:13:11,414 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632891358","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300056,"client":"192.168.2.21:43842","queuetimems":1106830,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:13:11,420 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632891374","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:43842","queuetimems":940536,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:13:11,434 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632891389","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.21:43842","queuetimems":1106755,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:13:24,250 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632904206","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.21:43842","queuetimems":939439,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:14:11,445 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632951398","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:43842","queuetimems":1166729,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:14:11,467 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605632951419","responsesize":"3313","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 2 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:43842","queuetimems":986651,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 17:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44825038, hits=37955535, hitRatio=84.67%, , cachingAccesses=38009996, cachingHits=37948059, cachingHitsRatio=99.84%, evictions=6384, evicted=0, evictedPerRun=0.0 2020-11-17 17:14:38,688 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 52465 ns. 2020-11-17 17:15:17,894 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 53442 ns. 2020-11-17 17:15:17,894 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionChecker average execution time: 21967 ns. 2020-11-17 17:16:11,464 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633071420","responsesize":"9929","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 6 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.22:36452","queuetimems":2,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":6} 2020-11-17 17:16:11,480 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633071420","responsesize":"4967","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 3 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300060,"client":"192.168.2.22:36452","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 17:16:11,486 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633071440","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x94\\xC4\\x0Acompliance\\xC4\\x0BJPE144830 \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.23:41484","queuetimems":20,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:16:38,747 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 41742 ns. 2020-11-17 17:17:11,462 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633131414","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:56658","queuetimems":59993,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:17:38,703 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 206809 ns. 2020-11-17 17:17:38,710 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 54595 ns. 2020-11-17 17:17:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 875349 ns. 2020-11-17 17:17:41,059 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":27,"starttimems":"1605633161007","responsesize":"3531","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 29 action(s) and 1st row key\u003d\\x02\\x07\\x92\\xC4\\x09snapshots\\xC4\\x06config","processingtimems":300052,"client":"192.168.2.21:56658","queuetimems":89583,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 17:17:41,467 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633161418","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:56658","queuetimems":89997,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:17:41,467 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633161414","responsesize":"6621","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 4 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.21:56658","queuetimems":89993,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":4} 2020-11-17 17:17:41,469 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633161418","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:56658","queuetimems":89518,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:18:11,471 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633191416","responsesize":"3313","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 2 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300055,"client":"192.168.2.21:56658","queuetimems":119994,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 17:18:11,473 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633191423","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:56658","queuetimems":55089,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:18:11,482 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633191435","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x90","processingtimems":300047,"client":"192.168.2.21:56658","queuetimems":120012,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:18:24,303 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633204253","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:56658","queuetimems":67918,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:19:11,493 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633251449","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.21:56658","queuetimems":129207,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:19:11,520 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633251468","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:56658","queuetimems":115027,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44825061, hits=37955558, hitRatio=84.67%, , cachingAccesses=38010019, cachingHits=37948082, cachingHitsRatio=99.84%, evictions=6414, evicted=0, evictedPerRun=0.0 2020-11-17 17:20:18,599 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionChecker average execution time: 20565 ns. 2020-11-17 17:20:18,600 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 36776 ns. 2020-11-17 17:20:38,665 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 62917 ns. 2020-11-17 17:21:11,509 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633371464","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.21:56658","queuetimems":249060,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:21:11,525 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633371481","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.21:56658","queuetimems":248936,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:21:11,538 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633371487","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:56658","queuetimems":235153,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:22:11,519 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: Can not complete this request in time, drop it: callId: 10 service: ClientService methodName: Mutate size: 372 connection: 192.168.2.23:51512 deadline: 9223372036854775807 param: region= aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., row=\x0B\x07\x93\xC4\x09snapshots\xC4\x06config\xC4\x0DrequestStatus connection: 192.168.2.23:51512 2020-11-17 17:22:11,572 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633131467","responsesize":"8278","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 5 action(s) and 1st row key\u003d\\x04\\x07\\x95\\xC4\\x0AztpService\\xC4\\x06device\\xC \u003cTRUNCATED\u003e","processingtimems":600105,"client":"192.168.2.21:56658","queuetimems":60045,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":5} 2020-11-17 17:22:38,747 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 44797 ns. 2020-11-17 17:22:41,514 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633461471","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x94\\xC4\\x0Acompliance\\xC4\\x0BJPE144830 \u003cTRUNCATED\u003e","processingtimems":300043,"client":"192.168.2.23:51512","queuetimems":30003,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:22:41,527 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633161424","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 2 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":600103,"client":"192.168.2.21:56658","queuetimems":89523,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 17:23:11,111 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633491061","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.22:45336","queuetimems":59592,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:23:11,518 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633491474","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.22:45336","queuetimems":60004,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:23:11,518 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633491473","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x94\\xC4\\x09BugAlerts\\xC4\\x06update\\xC4 \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.22:45336","queuetimems":60004,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:23:11,522 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633491473","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x95\\xC4\\x0AztpService\\xC4\\x06device\\xC \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:38152","queuetimems":60003,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:23:11,522 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633491471","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.22:45336","queuetimems":60001,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:23:11,530 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633491483","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:38152","queuetimems":60012,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:23:24,351 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633504304","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:38152","queuetimems":72834,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:23:38,708 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 223789 ns. 2020-11-17 17:23:38,710 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: nonceCleaner average execution time: 665686 ns. 2020-11-17 17:23:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 53214 ns. 2020-11-17 17:24:11,542 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633551495","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:38152","queuetimems":120024,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:24:11,564 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633551521","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300043,"client":"192.168.2.21:38152","queuetimems":120050,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44825418, hits=37955915, hitRatio=84.67%, , cachingAccesses=38010376, cachingHits=37948439, cachingHitsRatio=99.84%, evictions=6444, evicted=0, evictedPerRun=0.0 2020-11-17 17:25:27,791 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionChecker average execution time: 16297 ns. 2020-11-17 17:25:27,791 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 30932 ns. 2020-11-17 17:25:38,765 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 43791 ns. 2020-11-17 17:26:11,586 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633671539","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.22:50536","queuetimems":24,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:26:41,566 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605633701518","responsesize":"4975","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 4 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.22:50536","queuetimems":30002,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 17:26:41,570 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948989 2020-11-17 17:26:41,570 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633701526","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.21:44876","queuetimems":30009,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:27:11,570 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: Can not complete this request in time, drop it: callId: 9 service: ClientService methodName: Mutate size: 372 connection: 192.168.2.23:58628 deadline: 9223372036854775807 param: region= aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., row=\x0B\x07\x93\xC4\x09snapshots\xC4\x06config\xC4\x0DrequestStatus connection: 192.168.2.23:58628 2020-11-17 17:27:11,572 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948992 2020-11-17 17:27:38,749 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 59459 ns. 2020-11-17 17:27:41,578 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633761524","responsesize":"18200","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 11 action(s) and 1st row key\u003d\\x07\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ16292009 \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.23:58628","queuetimems":90006,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":11} 2020-11-17 17:27:41,580 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948991 2020-11-17 17:27:53,356 INFO [qtp368802354-71] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-17 17:28:11,573 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605633791522","responsesize":"1667","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 2 action(s) and 1st row key\u003d\\x02\\x07\\x92\\xC4\\x09snapshots\\xC4\\x06config","processingtimems":300051,"client":"192.168.2.21:44876","queuetimems":120004,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:28:11,573 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633791524","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:44876","queuetimems":120004,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:28:11,574 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948990 2020-11-17 17:28:11,579 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":14,"starttimems":"1605633791528","responsesize":"1771","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 15 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x0AztpService\\xC4\\x06config\\x \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:44876","queuetimems":120004,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:28:11,580 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633791531","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:44876","queuetimems":114472,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:28:11,583 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633791529","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.21:44876","queuetimems":114265,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:28:11,583 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633791528","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300055,"client":"192.168.2.21:44876","queuetimems":114363,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:28:19,515 INFO [qtp368802354-71] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-17 17:28:24,399 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633804352","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x99\\xC4\\x07Devices\\xC4\\x0BSSN19501010\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.23:58628","queuetimems":125531,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:28:30,608 INFO [qtp368802354-5778] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-17 17:29:05,460 INFO [qtp368802354-5779] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-17 17:29:11,614 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633851568","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":300046,"client":"192.168.2.21:49960","queuetimems":23,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:29:11,669 INFO [qtp368802354-316] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-17 17:29:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44825548, hits=37956045, hitRatio=84.68%, , cachingAccesses=38010506, cachingHits=37948569, cachingHitsRatio=99.84%, evictions=6474, evicted=0, evictedPerRun=0.0 2020-11-17 17:29:36,027 INFO [MobFileCache #0] mob.MobFileCache: MobFileCache Statistics, access: 0, miss: 0, hit: 0, hit ratio: 0%, evicted files: 0 2020-11-17 17:29:38,703 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 260897 ns. 2020-11-17 17:29:38,710 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: nonceCleaner average execution time: 613876 ns. 2020-11-17 17:29:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 40846 ns. 2020-11-17 17:30:27,829 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 70441 ns. 2020-11-17 17:30:27,829 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 26168 ns. at org.apache.hadoop.metrics2.impl.JmxCacheBuster.clearJmxCache(JmxCacheBuster.java:59) at org.apache.hadoop.hbase.regionserver.MetricsRegionAggregateSourceImpl$1.run(MetricsRegionAggregateSourceImpl.java:60) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-11-17 17:31:38,659 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 85259 ns. 2020-11-17 17:32:11,622 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: Can not complete this request in time, drop it: callId: 11 service: ClientService methodName: Mutate size: 372 connection: 192.168.2.23:39964 deadline: 9223372036854775807 param: region= aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., row=\x0B\x07\x93\xC4\x09snapshots\xC4\x06config\xC4\x0DrequestStatus connection: 192.168.2.23:39964 2020-11-17 17:32:11,622 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634031574","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x99\\xC4\\x07Devices\\xC4\\x0BSSN19501010\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.23:39964","queuetimems":30004,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:32:11,625 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948983 2020-11-17 17:32:11,630 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948993 2020-11-17 17:32:11,663 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633731573","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 2 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624\\ \u003cTRUNCATED\u003e","processingtimems":600090,"client":"192.168.2.23:58628","queuetimems":60059,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 17:32:11,669 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948984 2020-11-17 17:32:38,752 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 35215 ns. 2020-11-17 17:32:41,626 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":7,"starttimems":"1605634061580","responsesize":"1717","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 8 action(s) and 1st row key\u003d\\x0B\\x02\\x93\\xC4\\x04oidc\\xC4\\x08sessions\\xC4i74 \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.23:39964","queuetimems":60006,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:33:11,628 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634091574","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x90","processingtimems":300054,"client":"192.168.2.21:54064","queuetimems":90001,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:33:11,631 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634091584","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.23:39964","queuetimems":68125,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:33:11,651 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634091579","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624\\ \u003cTRUNCATED\u003e","processingtimems":300072,"client":"192.168.2.23:39964","queuetimems":68120,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:33:24,459 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634104401","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ18281992\\ \u003cTRUNCATED\u003e","processingtimems":300057,"client":"192.168.2.23:39964","queuetimems":80940,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:34:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44825615, hits=37956112, hitRatio=84.68%, , cachingAccesses=38010573, cachingHits=37948636, cachingHitsRatio=99.84%, evictions=6504, evicted=0, evictedPerRun=0.0 2020-11-17 17:35:28,395 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 53195 ns. 2020-11-17 17:35:28,395 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionChecker average execution time: 2956 ns. 2020-11-17 17:35:38,713 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 41266 ns. 2020-11-17 17:35:38,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 251766 ns. 2020-11-17 17:35:38,713 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: nonceCleaner average execution time: 626291 ns. at org.apache.hadoop.metrics2.impl.JmxCacheBuster.clearJmxCache(JmxCacheBuster.java:59) at org.apache.hadoop.hbase.regionserver.MetricsRegionAggregateSourceImpl$1.run(MetricsRegionAggregateSourceImpl.java:60) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-11-17 17:36:38,687 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 99923 ns. 2020-11-17 17:37:11,677 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: Can not complete this request in time, drop it: callId: 9 service: ClientService methodName: Mutate size: 372 connection: 192.168.2.23:49458 deadline: 9223372036854775807 param: region= aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., row=\x0B\x07\x93\xC4\x09snapshots\xC4\x06config\xC4\x0DrequestStatus connection: 192.168.2.23:49458 2020-11-17 17:37:11,678 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634331630","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ18281992\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.23:49458","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:38:11,682 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634091585","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 2 action(s) and 1st row key\u003d\\x07\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624\\ \u003cTRUNCATED\u003e","processingtimems":600096,"client":"192.168.2.23:39964","queuetimems":68126,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 17:38:11,688 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634091581","responsesize":"4970","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 3 action(s) and 1st row key\u003d\\x07\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ16292009\\ \u003cTRUNCATED\u003e","processingtimems":600106,"client":"192.168.2.23:39964","queuetimems":68121,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 17:38:11,696 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634391631","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x0BeventAlerts\\xC4\\x14alertman \u003cTRUNCATED\u003e","processingtimems":300064,"client":"192.168.2.23:49458","queuetimems":58654,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:38:24,508 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634404459","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ18295153\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.23:49458","queuetimems":20510,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:38:24,512 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948994 2020-11-17 17:38:24,512 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948995 2020-11-17 17:38:38,749 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 45707 ns. 2020-11-17 17:38:41,678 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634421631","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280689\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.23:49458","queuetimems":89962,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:39:11,784 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605633851547","responsesize":"6368","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 4 action(s) and 1st row key\u003d\\x04\\x07\\x95\\xC4\\x0AztpService\\xC4\\x06device\\xC \u003cTRUNCATED\u003e","processingtimems":900236,"client":"192.168.2.21:49960","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 17:39:11,784 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634151614","responsesize":"4970","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 3 action(s) and 1st row key\u003d\\x0B\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BHNN20205024\\ \u003cTRUNCATED\u003e","processingtimems":600170,"client":"192.168.2.23:39964","queuetimems":128152,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 17:39:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44825971, hits=37956468, hitRatio=84.68%, , cachingAccesses=38010929, cachingHits=37948992, cachingHitsRatio=99.84%, evictions=6534, evicted=0, evictedPerRun=0.0 2020-11-17 17:40:38,063 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 47195 ns. 2020-11-17 17:40:38,063 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionChecker average execution time: 13767 ns. 2020-11-17 17:41:11,804 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605633971586","responsesize":"81059","method":"Multi","param":"region\u003d aeris_v2,\\x06\\xCD\\x04\\x16\\x97\\xC4\\x05Smash\\xC4\\x07routing\\xC4\\x03pim\\xC4\\x0Asparsemode\\xC4\\x06status\\xC4\\x07default\\xC4\\x09routeIntf,160496158 \u003cTRUNCATED\u003e","processingtimems":900218,"client":"192.168.2.23:36412","queuetimems":120009,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":49} at org.apache.hadoop.metrics2.impl.JmxCacheBuster.clearJmxCache(JmxCacheBuster.java:59) at org.apache.hadoop.hbase.regionserver.MetricsRegionAggregateSourceImpl$1.run(MetricsRegionAggregateSourceImpl.java:60) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-11-17 17:41:38,703 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 260572 ns. 2020-11-17 17:41:38,710 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 38852 ns. 2020-11-17 17:41:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 906845 ns. 2020-11-17 17:41:40,342 INFO [qtp368802354-5781] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-17 17:41:57,718 INFO [qtp368802354-5812] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-17 17:42:06,507 INFO [qtp368802354-5817] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-17 17:42:11,811 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605634631682","responsesize":"1669","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 2 action(s) and 1st row key\u003d\\x02\\x07\\x92\\xC4\\x09snapshots\\xC4\\x06config","processingtimems":300129,"client":"192.168.2.23:58168","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:42:11,813 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634631683","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300130,"client":"192.168.2.22:42900","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:42:15,521 INFO [qtp368802354-5517] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-17 17:42:38,658 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 43893 ns. 2020-11-17 17:42:41,817 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1605634361670","responsesize":"52420","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 32 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ18281983 \u003cTRUNCATED\u003e","processingtimems":600147,"client":"192.168.2.23:49458","queuetimems":30040,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":30} 2020-11-17 17:43:11,724 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":23,"starttimems":"1605634091578","responsesize":"45329","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 31 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4 \u003cTRUNCATED\u003e","processingtimems":900145,"client":"192.168.2.21:54064","queuetimems":90004,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":8} 2020-11-17 17:43:11,727 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634091581","responsesize":"11589","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 7 action(s) and 1st row key\u003d\\x07\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BJPE18140464\\ \u003cTRUNCATED\u003e","processingtimems":900146,"client":"192.168.2.23:39964","queuetimems":68121,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":7} 2020-11-17 17:43:11,820 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634691694","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300126,"client":"192.168.2.21:42824","queuetimems":60010,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:43:11,821 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634691697","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300124,"client":"192.168.2.21:42824","queuetimems":60012,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:43:24,570 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634704518","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:42824","queuetimems":72825,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:43:34,098 INFO [qtp368802354-316] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-17 17:43:41,731 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634721686","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.21:42824","queuetimems":89991,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:43:41,732 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948997 2020-11-17 17:43:45,014 INFO [qtp368802354-5518] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-17 17:43:55,568 INFO [qtp368802354-5781] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-17 17:44:10,268 INFO [qtp368802354-5517] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-17 17:44:11,830 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634751784","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:42824","queuetimems":120089,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:44:11,833 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634751784","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:42824","queuetimems":120090,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:44:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44826309, hits=37956806, hitRatio=84.68%, , cachingAccesses=38011267, cachingHits=37949330, cachingHitsRatio=99.84%, evictions=6564, evicted=0, evictedPerRun=0.0 2020-11-17 17:44:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 48690 ns. 2020-11-17 17:45:38,236 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 40190 ns. 2020-11-17 17:45:38,236 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionChecker average execution time: 15109 ns. 2020-11-17 17:46:30,524 INFO [qtp368802354-5811] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null at org.apache.hadoop.metrics2.impl.JmxCacheBuster.clearJmxCache(JmxCacheBuster.java:59) at org.apache.hadoop.hbase.regionserver.MetricsRegionAggregateSourceImpl$1.run(MetricsRegionAggregateSourceImpl.java:60) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-11-17 17:47:11,867 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605634931821","responsesize":"1667","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 2 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:51100","queuetimems":2,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:47:11,870 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634931822","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.23:38548","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:47:11,877 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949002 2020-11-17 17:47:38,675 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 55767 ns. 2020-11-17 17:47:38,703 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 246119 ns. 2020-11-17 17:47:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 44073 ns. 2020-11-17 17:47:38,710 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: nonceCleaner average execution time: 568893 ns. 2020-11-17 17:47:41,876 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":3,"starttimems":"1605634961827","responsesize":"1683","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 4 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:51100","queuetimems":30005,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:47:41,882 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949001 2020-11-17 17:47:41,883 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949003 2020-11-17 17:47:41,883 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949009 2020-11-17 17:47:41,883 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948999 2020-11-17 17:47:41,883 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949008 2020-11-17 17:47:41,886 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948998 2020-11-17 17:48:11,378 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":7,"starttimems":"1605633791112","responsesize":"128881","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 79 action(s) and 1st row key\u003d\\x04\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280689 \u003cTRUNCATED\u003e","processingtimems":1500265,"client":"192.168.2.23:58628","queuetimems":119595,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":72} 2020-11-17 17:48:11,773 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634991725","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BJPE19362489\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.23:38548","queuetimems":59903,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:48:11,783 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":9,"starttimems":"1605634991729","responsesize":"1733","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 10 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x0AztpService\\xC4\\x06config\\x \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.21:51100","queuetimems":59905,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:48:11,785 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949007 2020-11-17 17:48:11,785 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949005 2020-11-17 17:48:11,785 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949000 2020-11-17 17:48:11,786 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949004 2020-11-17 17:48:11,786 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949006 2020-11-17 17:48:11,868 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634391655","responsesize":"4973","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 3 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624\\ \u003cTRUNCATED\u003e","processingtimems":900212,"client":"192.168.2.23:49458","queuetimems":7709,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 17:48:11,870 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605634691689","responsesize":"4711","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 3 action(s) and 1st row key\u003d\\x03\\x06\\x99\\xC4\\x07Devices\\xC4\\x0BSSN19501010\\ \u003cTRUNCATED\u003e","processingtimems":600181,"client":"192.168.2.23:58168","queuetimems":60006,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 17:48:11,871 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634991825","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BJPE16063289\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.23:38548","queuetimems":59996,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:48:11,877 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634991825","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280689\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.23:38548","queuetimems":60001,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:48:11,879 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605634391629","responsesize":"80800","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 49 action(s) and 1st row key\u003d\\x02\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BJPE16063289 \u003cTRUNCATED\u003e","processingtimems":900249,"client":"192.168.2.23:49458","queuetimems":59958,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":48} 2020-11-17 17:48:24,617 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635004571","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:51100","queuetimems":65818,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:48:41,778 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635021732","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:51100","queuetimems":82986,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:49:11,879 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635051834","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.21:51100","queuetimems":90010,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:49:11,885 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635051830","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300055,"client":"192.168.2.21:51100","queuetimems":112880,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:49:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44826569, hits=37957066, hitRatio=84.68%, , cachingAccesses=38011527, cachingHits=37949590, cachingHitsRatio=99.84%, evictions=6594, evicted=0, evictedPerRun=0.0 2020-11-17 17:49:38,763 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 45310 ns. 2020-11-17 17:50:47,778 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 45840 ns. 2020-11-17 17:50:47,779 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 21270 ns. 2020-11-17 17:51:11,919 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605634871810","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 2 action(s) and 1st row key\u003d\\x02\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624\\ \u003cTRUNCATED\u003e","processingtimems":600108,"client":"192.168.2.23:36868","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 17:52:11,929 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635231877","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280689\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.23:47476","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:52:41,965 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635261889","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624\\ \u003cTRUNCATED\u003e","processingtimems":300076,"client":"192.168.2.23:47476","queuetimems":30009,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:53:11,441 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1605635291380","responsesize":"4985","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 5 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x0AztpService\\xC4\\x06config\\xC \u003cTRUNCATED\u003e","processingtimems":300060,"client":"192.168.2.21:59418","queuetimems":59499,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 17:53:11,835 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605635291786","responsesize":"3321","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 3 action(s) and 1st row key\u003d\\x02\\x07\\x92\\xC4\\x09snapshots\\xC4\\x06config","processingtimems":300049,"client":"192.168.2.22:56054","queuetimems":59906,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 17:53:11,920 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635291870","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BJPE19362489\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.23:47476","queuetimems":59987,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:53:11,930 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635291872","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BJPE18170648\\ \u003cTRUNCATED\u003e","processingtimems":300057,"client":"192.168.2.23:47476","queuetimems":59988,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:53:11,931 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635291878","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BJPE19362489\\ \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.23:47476","queuetimems":59994,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:53:11,932 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635291880","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BJPE16161541\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.23:47476","queuetimems":59995,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:53:24,663 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635304618","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280689\\ \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.23:47476","queuetimems":72733,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:53:38,712 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 67008 ns. 2020-11-17 17:53:38,712 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 64751 ns. 2020-11-17 17:53:38,714 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 575057 ns. 2020-11-17 17:53:38,714 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 497152 ns. 2020-11-17 17:53:41,829 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635321778","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.23:47476","queuetimems":89896,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:53:41,829 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949011 2020-11-17 17:54:11,936 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635351881","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ18281983\\ \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.23:47476","queuetimems":119995,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:54:11,945 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635351886","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280689\\ \u003cTRUNCATED\u003e","processingtimems":300059,"client":"192.168.2.23:47476","queuetimems":120000,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:54:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44826825, hits=37957322, hitRatio=84.68%, , cachingAccesses=38011783, cachingHits=37949846, cachingHitsRatio=99.84%, evictions=6624, evicted=0, evictedPerRun=0.0 2020-11-17 17:55:38,747 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 40266 ns. 2020-11-17 17:55:48,219 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionChecker average execution time: 11621 ns. 2020-11-17 17:55:48,219 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 58703 ns. 2020-11-17 17:56:11,973 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635471924","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.22:60994","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:56:11,978 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949012 2020-11-17 17:56:11,978 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186948996 2020-11-17 17:57:11,981 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635231878","responsesize":"6624","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 4 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":600103,"client":"192.168.2.22:56054","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":4} 2020-11-17 17:57:11,983 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635531929","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.22:60994","queuetimems":60003,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:57:42,029 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635561974","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BHNN20205024\\ \u003cTRUNCATED\u003e","processingtimems":300055,"client":"192.168.2.23:57032","queuetimems":3,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:58:11,491 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635591442","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.23:57032","queuetimems":29471,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:58:11,885 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635291773","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 2 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":600112,"client":"192.168.2.21:59418","queuetimems":59895,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 17:58:11,898 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635591839","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300059,"client":"192.168.2.23:57032","queuetimems":29867,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:58:11,976 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635591931","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.22:34760","queuetimems":29958,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:58:11,978 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635591921","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x9A\\xC4\\x07Devices\\xC4\\x0BHSH13435130\\ \u003cTRUNCATED\u003e","processingtimems":300057,"client":"192.168.2.23:57032","queuetimems":29949,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:58:11,981 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635591932","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x9A\\xC4\\x07Devices\\xC4\\x0BHSH13435130\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.23:57032","queuetimems":29959,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:58:24,733 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":22,"starttimems":"1605635604664","responsesize":"3489","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 24 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x0AztpService\\xC4\\x06config\\x \u003cTRUNCATED\u003e","processingtimems":300069,"client":"192.168.2.21:40218","queuetimems":42690,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 17:58:41,897 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635621830","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ16421238\\ \u003cTRUNCATED\u003e","processingtimems":300066,"client":"192.168.2.23:57032","queuetimems":59857,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:59:11,989 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635651937","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x0BeventAlerts\\xC4\\x14alertman \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.22:34760","queuetimems":60096,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:59:12,010 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635651946","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300064,"client":"192.168.2.22:34760","queuetimems":89671,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 17:59:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44826915, hits=37957412, hitRatio=84.68%, , cachingAccesses=38011873, cachingHits=37949936, cachingHitsRatio=99.84%, evictions=6654, evicted=0, evictedPerRun=0.0 2020-11-17 17:59:38,744 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 53523 ns. 2020-11-17 17:59:38,745 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: nonceCleaner average execution time: 836140 ns. 2020-11-17 17:59:38,745 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 45606 ns. 2020-11-17 17:59:38,745 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 211748 ns. 2020-11-17 18:00:38,751 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 47688 ns. 2020-11-17 18:00:48,462 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 41622 ns. 2020-11-17 18:00:48,462 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionChecker average execution time: 19283 ns. 2020-11-17 18:01:12,028 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635771979","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x94\\xC4\\x09BugAlerts\\xC4\\x06update\\xC4 \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.22:39144","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:02:12,033 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635831989","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x9A\\xC4\\x07Devices\\xC4\\x0BJPE15500897\\ \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.23:36716","queuetimems":2,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:02:12,038 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: Can not complete this request in time, drop it: callId: 11 service: ClientService methodName: Mutate size: 372 connection: 192.168.2.23:36716 deadline: 9223372036854775807 param: region= aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., row=\x0B\x07\x93\xC4\x09snapshots\xC4\x06config\xC4\x0DrequestStatus connection: 192.168.2.23:36716 2020-11-17 18:02:42,079 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635862032","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x95\\xC4\\x0AztpService\\xC4\\x06device\\xC \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:47922","queuetimems":30045,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:03:11,544 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635891493","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BJPE16161541\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.23:36716","queuetimems":59506,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:03:11,934 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635891886","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x90","processingtimems":300047,"client":"192.168.2.21:47922","queuetimems":59898,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:03:11,954 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635891899","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300055,"client":"192.168.2.21:47922","queuetimems":59910,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:03:12,024 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635891977","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ16421238\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.23:36716","queuetimems":59989,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:03:12,028 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":21,"starttimems":"1605635291871","responsesize":"13411","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 29 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x0AztpService\\xC4\\x06config\\x \u003cTRUNCATED\u003e","processingtimems":900157,"client":"192.168.2.21:59418","queuetimems":59991,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":8} 2020-11-17 18:03:12,030 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605635591935","responsesize":"4978","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 4 action(s) and 1st row key\u003d\\x02\\x07\\x92\\xC4\\x09snapshots\\xC4\\x06config","processingtimems":600095,"client":"192.168.2.22:34760","queuetimems":29962,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 18:03:12,038 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635891980","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280689\\ \u003cTRUNCATED\u003e","processingtimems":300058,"client":"192.168.2.23:36716","queuetimems":59991,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:03:12,039 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635891982","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":300057,"client":"192.168.2.21:47922","queuetimems":59992,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:03:12,044 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949010 2020-11-17 18:03:24,790 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635904736","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.21:47922","queuetimems":72745,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:03:41,949 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635921899","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:47922","queuetimems":89908,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:04:12,035 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635951991","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300044,"client":"192.168.2.21:47922","queuetimems":112040,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:04:12,059 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605635952011","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:47922","queuetimems":112054,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:04:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44827150, hits=37957647, hitRatio=84.68%, , cachingAccesses=38012108, cachingHits=37950171, cachingHitsRatio=99.84%, evictions=6684, evicted=0, evictedPerRun=0.0 2020-11-17 18:05:38,660 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 53367 ns. 2020-11-17 18:05:38,705 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 231774 ns. 2020-11-17 18:05:38,710 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: nonceCleaner average execution time: 826549 ns. 2020-11-17 18:05:38,748 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 45588 ns. 2020-11-17 18:05:58,704 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 61591 ns. 2020-11-17 18:05:58,704 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 25181 ns. 2020-11-17 18:06:12,070 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636072028","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300042,"client":"192.168.2.21:47922","queuetimems":209989,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:06:38,751 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 42318 ns. 2020-11-17 18:07:12,085 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636132033","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:47922","queuetimems":269991,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:07:12,088 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949013 2020-11-17 18:07:12,089 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636132041","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:47922","queuetimems":269998,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:07:42,126 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636162081","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.21:47922","queuetimems":322129,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:08:11,596 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636191545","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:47922","queuetimems":329510,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:08:12,000 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636191940","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300060,"client":"192.168.2.21:47922","queuetimems":275658,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:08:12,006 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636191955","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:47922","queuetimems":261881,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:08:12,073 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636192026","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:47922","queuetimems":329991,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:08:12,073 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636192031","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300042,"client":"192.168.2.21:47922","queuetimems":329991,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:08:12,078 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636192031","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:47922","queuetimems":261951,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:08:12,087 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636192039","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:47922","queuetimems":329998,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:08:12,091 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":9,"starttimems":"1605636192045","responsesize":"3387","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 11 action(s) and 1st row key\u003d\\x03\\x07\\x93\\xC4\\x0AztpService\\xC4\\x06config\\x \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:47922","queuetimems":329999,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 18:08:24,842 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636204790","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300052,"client":"192.168.2.21:47922","queuetimems":339152,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:08:42,017 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636221949","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300068,"client":"192.168.2.21:47922","queuetimems":291767,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:09:12,081 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636252037","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":300044,"client":"192.168.2.21:47922","queuetimems":386399,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:09:12,108 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636252060","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:47922","queuetimems":321851,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:09:12,113 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949016 2020-11-17 18:09:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44827398, hits=37957895, hitRatio=84.68%, , cachingAccesses=38012356, cachingHits=37950419, cachingHitsRatio=99.84%, evictions=6714, evicted=0, evictedPerRun=0.0 2020-11-17 18:10:38,669 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 64553 ns. 2020-11-17 18:11:08,268 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionChecker average execution time: 21990 ns. 2020-11-17 18:11:08,268 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 40405 ns. 2020-11-17 18:11:12,131 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636372077","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x94\\xC4\\x09BugAlerts\\xC4\\x06update\\xC4 \u003cTRUNCATED\u003e","processingtimems":300054,"client":"192.168.2.22:52186","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:11:38,707 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 240739 ns. 2020-11-17 18:11:38,710 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 50151 ns. 2020-11-17 18:11:38,710 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 544716 ns. 2020-11-17 18:12:12,140 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636432091","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.23:52682","queuetimems":59872,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:12:38,746 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 39052 ns. 2020-11-17 18:12:42,178 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636462132","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.22:53626","queuetimems":1,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:13:11,647 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636491598","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.22:53626","queuetimems":29466,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:13:11,648 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949017 2020-11-17 18:13:11,649 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949020 2020-11-17 18:13:12,054 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636492004","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x95\\xC4\\x0AztpService\\xC4\\x06device\\xC \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:37122","queuetimems":29872,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:13:12,056 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949018 2020-11-17 18:13:12,056 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] regionserver.RSRpcServices: Client tried to access missing scanner 3879920799186949019 2020-11-17 18:13:12,056 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636492007","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x07\\x94\\xC4\\x0Acompliance\\xC4\\x0BJPE144830 \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.23:55166","queuetimems":29875,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:13:12,134 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636492075","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300059,"client":"192.168.2.22:53626","queuetimems":29942,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:13:12,140 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636492092","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.23:55166","queuetimems":29272,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:13:24,904 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636504844","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300059,"client":"192.168.2.23:55166","queuetimems":41824,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:13:42,079 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636522019","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x93\\xC4\\x08auditLog\\xC4\\x07indexes\\xC4 \u003cTRUNCATED\u003e","processingtimems":300060,"client":"192.168.2.22:53626","queuetimems":59885,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:14:12,149 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636552093","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624\\ \u003cTRUNCATED\u003e","processingtimems":300056,"client":"192.168.2.23:57616","queuetimems":6,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:14:12,184 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636552118","responsesize":"18200","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 11 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17250690 \u003cTRUNCATED\u003e","processingtimems":300065,"client":"192.168.2.23:57616","queuetimems":29,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":11} 2020-11-17 18:14:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44827524, hits=37958021, hitRatio=84.68%, , cachingAccesses=38012482, cachingHits=37950545, cachingHitsRatio=99.84%, evictions=6744, evicted=0, evictedPerRun=0.0 2020-11-17 18:15:38,724 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 79641 ns. 2020-11-17 18:16:12,177 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636672132","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 1 action(s) and 1st row key\u003d\\x02\\x07\\x90","processingtimems":300045,"client":"192.168.2.21:39536","queuetimems":120019,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:16:18,766 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionChecker average execution time: 20316 ns. 2020-11-17 18:16:18,766 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 37442 ns. 2020-11-17 18:17:38,704 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 523721 ns. 2020-11-17 18:17:38,710 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 42739 ns. 2020-11-17 18:17:38,710 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: nonceCleaner average execution time: 536246 ns. 2020-11-17 18:17:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 57568 ns. 2020-11-17 18:18:11,792 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636791650","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300141,"client":"192.168.2.21:39536","queuetimems":230733,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:18:12,103 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636792057","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300046,"client":"192.168.2.21:39536","queuetimems":231141,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:18:12,107 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636792057","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:39536","queuetimems":231139,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:18:12,186 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636792141","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.21:39536","queuetimems":231117,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:18:12,189 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636792134","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300055,"client":"192.168.2.21:39536","queuetimems":231127,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:18:24,958 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636804905","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.21:39536","queuetimems":243859,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:18:42,135 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636822082","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.21:39536","queuetimems":261161,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:19:12,201 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636852151","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:39536","queuetimems":291092,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:19:12,230 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636852185","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300045,"client":"192.168.2.21:39536","queuetimems":291132,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:19:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44827742, hits=37958239, hitRatio=84.68%, , cachingAccesses=38012700, cachingHits=37950763, cachingHitsRatio=99.84%, evictions=6774, evicted=0, evictedPerRun=0.0 2020-11-17 18:21:12,230 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636972177","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300053,"client":"192.168.2.21:39536","queuetimems":411118,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:21:28,361 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 40654 ns. 2020-11-17 18:21:28,361 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactionChecker average execution time: 16378 ns. 2020-11-17 18:21:38,701 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 49692 ns. 2020-11-17 18:22:12,238 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605636732141","responsesize":"6632","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 5 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":600097,"client":"192.168.2.21:39536","queuetimems":180027,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":4} 2020-11-17 18:22:38,747 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: cvp21,16201,1605569374704-HeapMemoryTunerChore average execution time: 48345 ns. 2020-11-17 18:22:42,291 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":23,"starttimems":"1605636762179","responsesize":"5154","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 26 action(s) and 1st row key\u003d\\x04\\x07\\x95\\xC4\\x0AztpService\\xC4\\x06device\\x \u003cTRUNCATED\u003e","processingtimems":600112,"client":"192.168.2.21:39536","queuetimems":210088,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":3} 2020-11-17 18:23:12,153 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637092103","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:39536","queuetimems":530982,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:23:12,158 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637092109","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300049,"client":"192.168.2.21:39536","queuetimems":505507,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:23:12,230 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":23,"starttimems":"1605636492080","responsesize":"43671","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 30 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":900150,"client":"192.168.2.21:37122","queuetimems":29947,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":7} 2020-11-17 18:23:12,246 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637092191","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300055,"client":"192.168.2.21:39536","queuetimems":455083,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:23:12,249 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637092191","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4( \u003cTRUNCATED\u003e","processingtimems":300058,"client":"192.168.2.21:39536","queuetimems":441154,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:23:12,260 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605636492089","responsesize":"246482","method":"Multi","param":"region\u003d aeris_v2,\\x06\\xCD\\x04\\x16\\x97\\xC4\\x05Smash\\xC4\\x07routing\\xC4\\x03pim\\xC4\\x0Asparsemode\\xC4\\x06status\\xC4\\x07default\\xC4\\x09routeIntf,160496158 \u003cTRUNCATED\u003e","processingtimems":900170,"client":"192.168.2.23:54342","queuetimems":29925,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":149} 2020-11-17 18:23:25,010 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637104959","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:39536","queuetimems":453919,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:23:38,711 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: MovedRegionsCleaner for region cvp21,16201,1605569374704 average execution time: 61324 ns. 2020-11-17 18:23:38,712 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: CompactedHFilesCleaner average execution time: 902487 ns. 2020-11-17 18:23:38,713 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: nonceCleaner average execution time: 870138 ns. 2020-11-17 18:23:42,185 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637122137","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300048,"client":"192.168.2.21:39536","queuetimems":485191,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:24:12,250 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637152203","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300047,"client":"192.168.2.21:39536","queuetimems":501022,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:24:12,281 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637152231","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x95\\xC4\\x07Devices\\xC4\\x0BHSH13465064\\ \u003cTRUNCATED\u003e","processingtimems":300050,"client":"192.168.2.21:39536","queuetimems":514983,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:24:36,024 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.72 GB, freeSize=111.67 MB, max=1.83 GB, blockCount=45899, accesses=44827750, hits=37958247, hitRatio=84.68%, , cachingAccesses=38012708, cachingHits=37950771, cachingHitsRatio=99.84%, evictions=6804, evicted=0, evictedPerRun=0.0 2020-11-17 18:27:31,225 WARN [main-SendThread(cvp21:2181)] zookeeper.ClientCnxn: Client session timed out, have not heard from server in 103990ms for sessionid 0x1000be3d4bf6c56 2020-11-17 18:27:31,240 INFO [regionserver/cvp21:16201.Chore.1] hbase.ChoreService: Chore [ScheduledChore: Name: MemstoreFlusherChore Period: 10000 Unit: MILLISECONDS] is enabled. 2020-11-17 18:27:31,239 WARN [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 99564ms No GCs detected 2020-11-17 18:27:31,236 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605636432089","responsesize":"64263","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 39 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624 \u003cTRUNCATED\u003e","processingtimems":1219146,"client":"192.168.2.23:52682","queuetimems":60010,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":38} 2020-11-17 18:27:31,235 WARN [regionserver/cvp21:16201] util.Sleeper: We slept 103620ms instead of 5000ms, this is likely due to a long garbage collecting pause and it's usually bad, see http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired 2020-11-17 18:27:31,235 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637332240","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":318994,"client":"192.168.2.21:39536","queuetimems":681198,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:27:31,235 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637272234","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x04,1598623754299.73aae5715542e1a5370f7540d2117abc., for 1 action(s) and 1st row key\u003d\\x04\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":379000,"client":"192.168.2.21:39536","queuetimems":621196,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:27:31,241 INFO [regionserver/cvp21:16201.Chore.2] hbase.ChoreService: Chore [ScheduledChore: Name: CompactionChecker Period: 10000 Unit: MILLISECONDS] is enabled. 2020-11-17 18:27:31,241 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: Chore: MemstoreFlusherChore missed its start time 2020-11-17 18:27:31,241 INFO [main-SendThread(cvp21:2181)] zookeeper.ClientCnxn: Client session timed out, have not heard from server in 103990ms for sessionid 0x1000be3d4bf6c56, closing socket connection and attempting reconnect 2020-11-17 18:27:31,249 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: Chore: CompactionChecker missed its start time 2020-11-17 18:27:31,251 INFO [regionserver/cvp21:16201.Chore.3] hbase.ChoreService: Chore [ScheduledChore: Name: CompactionThroughputTuner Period: 60000 Unit: MILLISECONDS] is enabled. 2020-11-17 18:27:31,251 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: Chore: CompactionThroughputTuner missed its start time 2020-11-17 18:27:31,252 INFO [regionserver/cvp21:16201.Chore.4] hbase.ScheduledChore: CompactionChecker average execution time: 13796 ns. 2020-11-17 18:27:31,252 INFO [regionserver/cvp21:16201.Chore.3] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 71188 ns. 2020-11-17 18:27:31,252 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: MemstoreFlusherChore average execution time: 52265 ns. 2020-11-17 18:27:31,253 INFO [regionserver/cvp21:16201.Chore.1] hbase.ChoreService: Chore [ScheduledChore: Name: cvp21,16201,1605569374704-HeapMemoryTunerChore Period: 60000 Unit: MILLISECONDS] is enabled. 2020-11-17 18:27:31,253 INFO [regionserver/cvp21:16201.Chore.1] hbase.ScheduledChore: Chore: cvp21,16201,1605569374704-HeapMemoryTunerChore missed its start time 2020-11-17 18:27:31,264 INFO [qtp368802354-5781] hfile.CacheConfig: Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false with blockCache=null 2020-11-17 18:27:31,317 ERROR [regionserver/cvp21:16201] regionserver.HRegionServer: ***** ABORTING region server cvp21,16201,1605569374704: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing cvp21,16201,1605569374704 as dead server at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:366) at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:253) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:519) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15331) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318) ***** org.apache.hadoop.hbase.YouAreDeadException: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing cvp21,16201,1605569374704 as dead server at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:366) at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:253) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:519) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15331) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:99) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:89) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:341) at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:1260) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1055) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.YouAreDeadException): org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing cvp21,16201,1605569374704 as dead server at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:366) at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:253) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:519) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15331) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:389) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:97) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:423) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:419) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:796) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:427) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:328) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2020-11-17 18:27:31,321 ERROR [regionserver/cvp21:16201] regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [org.apache.hadoop.hbase.coprocessor.AggregateImplementation, org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint] 2020-11-17 18:27:31,340 INFO [regionserver/cvp21:16201] regionserver.HRegionServer: , "Bulkload_min": 0, "Bulkload_max": 0, "Bulkload_mean": 0, "Bulkload_25th_percentile": 0, "Bulkload_median": 0, "Bulkload_75th_percentile": 0, "Bulkload_90th_percentile": 0, "Bulkload_95th_percentile": 0, "Bulkload_98th_percentile": 0, "Bulkload_99th_percentile": 0, "Bulkload_99.9th_percentile": 0, "ServerReadQueryPerSecond_count": 6307488, "ServerReadQueryPerSecond_mean_rate": 92.386776983498, "ServerReadQueryPerSecond_1min_rate": 0.0013686068361525186, "ServerReadQueryPerSecond_5min_rate": 0.028984982362780784, "ServerReadQueryPerSecond_15min_rate": 0.12322685045932673, "ServerWriteQueryPerSecond_count": 93657832, "ServerWriteQueryPerSecond_mean_rate": 1371.821117730287, "ServerWriteQueryPerSecond_1min_rate": 0.03722814864288689, "ServerWriteQueryPerSecond_5min_rate": 0.22863523233066452, "ServerWriteQueryPerSecond_15min_rate": 0.20951058576224862 } ] } 2020-11-17 18:27:31,361 INFO [regionserver/cvp21:16201] regionserver.HRegionServer: ***** STOPPING region server 'cvp21,16201,1605569374704' ***** 2020-11-17 18:27:31,361 INFO [regionserver/cvp21:16201] regionserver.HRegionServer: STOPPED: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing cvp21,16201,1605569374704 as dead server at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:366) at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:253) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:519) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15331) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318) 2020-11-17 18:27:31,363 INFO [regionserver/cvp21:16201] regionserver.SplitLogWorker: Sending interrupt to stop the worker thread 2020-11-17 18:27:31,363 INFO [regionserver/cvp21:16201] regionserver.HRegionServer: Stopping infoServer 2020-11-17 18:27:31,363 INFO [SplitLogWorker-cvp21:16201] regionserver.SplitLogWorker: SplitLogWorker interrupted. Exiting. 2020-11-17 18:27:31,366 INFO [SplitLogWorker-cvp21:16201] regionserver.SplitLogWorker: SplitLogWorker cvp21,16201,1605569374704 exiting 2020-11-17 18:27:31,391 INFO [regionserver/cvp21:16201] handler.ContextHandler: Stopped o.e.j.w.WebAppContext@7f572c37{/,null,UNAVAILABLE}{file:/cvpi/hbase/hbase-webapps/regionserver} 2020-11-17 18:27:31,402 INFO [regionserver/cvp21:16201] server.AbstractConnector: Stopped ServerConnector@44841b43{HTTP/1.1,[http/1.1]}{0.0.0.0:16301} 2020-11-17 18:27:31,402 INFO [regionserver/cvp21:16201] handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@58d63b16{/static,file:///cvpi/hbase/hbase-webapps/static/,UNAVAILABLE} 2020-11-17 18:27:31,402 INFO [regionserver/cvp21:16201] handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@1dc2de84{/logs,file:///cvpi/hbase/logs/,UNAVAILABLE} 2020-11-17 18:27:31,722 INFO [regionserver/cvp21:16201] regionserver.HeapMemoryManager: Stopping 2020-11-17 18:27:31,722 INFO [regionserver/cvp21:16201] flush.RegionServerFlushTableProcedureManager: Stopping region server flush procedure manager abruptly. 2020-11-17 18:27:31,723 INFO [regionserver/cvp21:16201] snapshot.RegionServerSnapshotManager: Stopping RegionServerSnapshotManager abruptly. 2020-11-17 18:27:31,723 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher: MemStoreFlusher.0 exiting 2020-11-17 18:27:31,723 INFO [MemStoreFlusher.1] regionserver.MemStoreFlusher: MemStoreFlusher.1 exiting 2020-11-17 18:27:31,725 INFO [regionserver/cvp21:16201] regionserver.HRegionServer: aborting server cvp21,16201,1605569374704 2020-11-17 18:27:31,728 INFO [regionserver/cvp21:16201] regionserver.HRegionServer: Waiting on 9 regions to close 2020-11-17 18:27:31,736 ERROR [RS_CLOSE_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Memstore data size is 43067065 2020-11-17 18:27:31,739 ERROR [RS_CLOSE_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Memstore data size is 3063759 2020-11-17 18:27:31,738 ERROR [RS_CLOSE_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Memstore data size is 88704041 2020-11-17 18:27:31,741 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Closed aeris_v2,\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e. 2020-11-17 18:27:31,741 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Closed aeris_v2,\x11\xCD\x04\x0C\x96\xC4\x05Smash\xC4\x08routing6\xC4\x03pim\xC4\x06status\xC4\x07default\xC4\x14intfSecondaryAddress,1604670627883.82421ce8c5c8b48c573baee83caa8305. 2020-11-17 18:27:31,741 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Closed aeris_v2,\x04,1598623754299.73aae5715542e1a5370f7540d2117abc. 2020-11-17 18:27:31,754 INFO [main-SendThread(cvp23:2181)] zookeeper.ClientCnxn: Opening socket connection to server cvp23/192.168.2.23:2181. Will not attempt to authenticate using SASL (unknown error) 2020-11-17 18:27:31,755 INFO [main-SendThread(cvp23:2181)] zookeeper.ClientCnxn: Socket connection established to cvp23/192.168.2.23:2181, initiating session 2020-11-17 18:27:31,762 WARN [main-SendThread(cvp23:2181)] zookeeper.ClientCnxn: Unable to reconnect to ZooKeeper service, session 0x1000be3d4bf6c56 has expired 2020-11-17 18:27:31,762 INFO [main-SendThread(cvp23:2181)] zookeeper.ClientCnxn: Unable to reconnect to ZooKeeper service, session 0x1000be3d4bf6c56 has expired, closing socket connection 2020-11-17 18:27:31,762 ERROR [main-EventThread] regionserver.HRegionServer: ***** ABORTING region server cvp21,16201,1605569374704: regionserver:16201-0x1000be3d4bf6c56, quorum=cvp21:2181,cvp22:2181,cvp23:2181, baseZNode=/hbase regionserver:16201-0x1000be3d4bf6c56 received expired from ZooKeeper, aborting ***** org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired at org.apache.hadoop.hbase.zookeeper.ZKWatcher.connectionEvent(ZKWatcher.java:556) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:486) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:40) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505) 2020-11-17 18:27:31,762 ERROR [main-EventThread] regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [org.apache.hadoop.hbase.coprocessor.AggregateImplementation, org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint] 2020-11-17 18:27:31,765 INFO [main-EventThread] regionserver.HRegionServer: , "Bulkload_min": 0, "Bulkload_max": 0, "Bulkload_mean": 0, "Bulkload_25th_percentile": 0, "Bulkload_median": 0, "Bulkload_75th_percentile": 0, "Bulkload_90th_percentile": 0, "Bulkload_95th_percentile": 0, "Bulkload_98th_percentile": 0, "Bulkload_99th_percentile": 0, "Bulkload_99.9th_percentile": 0, "ServerReadQueryPerSecond_count": 6307488, "ServerReadQueryPerSecond_mean_rate": 92.386776983498, "ServerReadQueryPerSecond_1min_rate": 0.0013686068361525186, "ServerReadQueryPerSecond_5min_rate": 0.028984982362780784, "ServerReadQueryPerSecond_15min_rate": 0.12322685045932673, "ServerWriteQueryPerSecond_count": 93657832, "ServerWriteQueryPerSecond_mean_rate": 1371.821117730287, "ServerWriteQueryPerSecond_1min_rate": 0.03722814864288689, "ServerWriteQueryPerSecond_5min_rate": 0.22863523233066452, "ServerWriteQueryPerSecond_15min_rate": 0.20951058576224862 } ] } 2020-11-17 18:27:31,767 ERROR [main-EventThread] regionserver.HRegionServer: ***** ABORTING region server cvp21,16201,1605569374704: regionserver:16201-0x1000be3d4bf6c56, quorum=cvp21:2181,cvp22:2181,cvp23:2181, baseZNode=/hbase regionserver:16201-0x1000be3d4bf6c56 received expired from ZooKeeper, aborting ***** org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired at org.apache.hadoop.hbase.zookeeper.ZKWatcher.connectionEvent(ZKWatcher.java:556) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:486) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505) 2020-11-17 18:27:31,767 ERROR [main-EventThread] regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [org.apache.hadoop.hbase.coprocessor.AggregateImplementation, org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint] 2020-11-17 18:27:31,769 INFO [main-EventThread] regionserver.HRegionServer: , "Bulkload_min": 0, "Bulkload_max": 0, "Bulkload_mean": 0, "Bulkload_25th_percentile": 0, "Bulkload_median": 0, "Bulkload_75th_percentile": 0, "Bulkload_90th_percentile": 0, "Bulkload_95th_percentile": 0, "Bulkload_98th_percentile": 0, "Bulkload_99th_percentile": 0, "Bulkload_99.9th_percentile": 0, "ServerReadQueryPerSecond_count": 6307488, "ServerReadQueryPerSecond_mean_rate": 92.386776983498, "ServerReadQueryPerSecond_1min_rate": 0.0013686068361525186, "ServerReadQueryPerSecond_5min_rate": 0.028984982362780784, "ServerReadQueryPerSecond_15min_rate": 0.12322685045932673, "ServerWriteQueryPerSecond_count": 93657832, "ServerWriteQueryPerSecond_mean_rate": 1371.821117730287, "ServerWriteQueryPerSecond_1min_rate": 0.03722814864288689, "ServerWriteQueryPerSecond_5min_rate": 0.22863523233066452, "ServerWriteQueryPerSecond_15min_rate": 0.20951058576224862 } ] } 2020-11-17 18:27:31,770 INFO [main-EventThread] zookeeper.ClientCnxn: EventThread shut down for session: 0x1000be3d4bf6c56 2020-11-17 18:27:32,729 INFO [regionserver/cvp21:16201] regionserver.HRegionServer: Waiting on 6 regions to close 2020-11-17 18:27:38,704 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: Chore: CompactedHFilesCleaner was stopped 2020-11-17 18:27:38,710 INFO [regionserver/cvp21:16201.Chore.5] hbase.ScheduledChore: Chore: MovedRegionsCleaner for region cvp21,16201,1605569374704 was stopped 2020-11-17 18:27:41,225 INFO [regionserver/cvp21:16201.leaseChecker] regionserver.Leases: Closed leases 2020-11-17 18:27:41,243 INFO [regionserver/cvp21:16201.Chore.5] hbase.ScheduledChore: Chore: MemstoreFlusherChore was stopped 2020-11-17 18:27:41,249 INFO [regionserver/cvp21:16201.Chore.4] hbase.ScheduledChore: Chore: CompactionChecker was stopped 2020-11-17 18:27:43,226 WARN [RpcServer.default.FPBQ.Fifo.handler=13,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637362294","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300932,"client":"192.168.2.21:39536","queuetimems":711052,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:28:12,231 WARN [RpcServer.default.FPBQ.Fifo.handler=9,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637091792","responsesize":"3316","method":"Multi","param":"region\u003d aeris_v2,\\x02,1598623754299.e5dc5c620be19b02666a19c280ff582e., for 2 action(s) and 1st row key\u003d\\x02\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":600439,"client":"192.168.2.21:39536","queuetimems":505189,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 18:28:12,231 WARN [RpcServer.default.FPBQ.Fifo.handler=11,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":1,"starttimems":"1605636492078","responsesize":"44413","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 27 action(s) and 1st row key\u003d\\x03\\x06\\x97\\xC4\\x07Devices\\xC4\\x0BSSJ17280624 \u003cTRUNCATED\u003e","processingtimems":1200153,"client":"192.168.2.23:55166","queuetimems":29944,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":26} 2020-11-17 18:28:12,231 WARN [RpcServer.default.FPBQ.Fifo.handler=8,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637392160","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300071,"client":"192.168.2.21:39536","queuetimems":741015,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:28:12,234 WARN [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637392153","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300081,"client":"192.168.2.21:39536","queuetimems":741024,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:28:13,232 WARN [RpcServer.default.FPBQ.Fifo.handler=15,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637392261","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300971,"client":"192.168.2.21:39536","queuetimems":651012,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:28:13,234 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637392232","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":301000,"client":"192.168.2.21:39536","queuetimems":741065,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:28:13,236 WARN [RpcServer.default.FPBQ.Fifo.handler=5,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637392252","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300983,"client":"192.168.2.21:39536","queuetimems":651092,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:28:13,247 WARN [RpcServer.default.FPBQ.Fifo.handler=7,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637392247","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x94\\xC4\\x08Turbines\\xC4\\x06config\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":301000,"client":"192.168.2.21:39536","queuetimems":715523,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:28:25,231 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637405012","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300219,"client":"192.168.2.21:39536","queuetimems":663746,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:28:42,236 WARN [RpcServer.default.FPBQ.Fifo.handler=6,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637422185","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1., for 1 action(s) and 1st row key\u003d\\x09\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300051,"client":"192.168.2.21:39536","queuetimems":771010,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:29:12,259 WARN [RpcServer.default.FPBQ.Fifo.handler=3,queue=1,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637452250","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300008,"client":"192.168.2.21:39536","queuetimems":710963,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:29:13,243 WARN [RpcServer.default.FPBQ.Fifo.handler=14,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637452283","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79., for 1 action(s) and 1st row key\u003d\\x07\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300960,"client":"192.168.2.21:39536","queuetimems":801103,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:29:13,252 ERROR [RS_CLOSE_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Memstore data size is 52785469 2020-11-17 18:29:13,254 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Closed aeris_v2,\x07,1598623754299.bdaa31a63b7f3d06d9909b7560333e79. 2020-11-17 18:29:13,259 ERROR [RS_CLOSE_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Memstore data size is 38581084 2020-11-17 18:29:13,261 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Closed aeris_v2,\x06\xCD\x04\x16\x97\xC4\x05Smash\xC4\x07routing\xC4\x03pim\xC4\x0Asparsemode\xC4\x06status\xC4\x07default\xC4\x09routeIntf,1604961588305.fc93f6dcc564341dcc983bf30971f73d. 2020-11-17 18:29:13,264 ERROR [RS_CLOSE_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Memstore data size is 793312 2020-11-17 18:29:13,264 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Closed aeris_v2,\x09,1598623754299.5e5f05205d7e40b0141cd977dd009ba1. 2020-11-17 18:29:13,269 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-2] regionserver.HRegion: Closed __test-table_cvp23__,,1598623751841.bac2756ecc315bfd457fb71d81dd3eb5. 2020-11-17 18:29:13,279 INFO [regionserver/cvp21:16201] regionserver.HRegionServer: Waiting on 2 regions to close 2020-11-17 18:29:39,182 INFO [regionserver/cvp21:16201.Chore.2] hbase.ScheduledChore: Chore: nonceCleaner was stopped 2020-11-17 18:32:31,255 INFO [regionserver/cvp21:16201.Chore.4] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 49799 ns. 2020-11-17 18:32:31,315 WARN [RpcServer.default.FPBQ.Fifo.handler=12,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637651256","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 1 action(s) and 1st row key\u003d\\x0B\\x06\\x94\\xC4\\x07Devices\\xC4 B69CD37CFB7C1C9 \u003cTRUNCATED\u003e","processingtimems":300058,"client":"192.168.2.21:39536","queuetimems":924189,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:32:31,319 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637651256","responsesize":"1659","method":"Multi","param":"region\u003d aeris_v2,\\x03,1598623754299.0469ce862509756add4a8cdb00bc465c., for 1 action(s) and 1st row key\u003d\\x03\\x06\\x93\\xC4\\x08Turbines\\xC4\\x06status\\xC4\\ \u003cTRUNCATED\u003e","processingtimems":300063,"client":"192.168.2.21:39536","queuetimems":924027,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1} 2020-11-17 18:32:31,324 ERROR [RS_CLOSE_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Memstore data size is 78719502 2020-11-17 18:32:31,324 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-1] regionserver.HRegion: Closed aeris_v2,\x03,1598623754299.0469ce862509756add4a8cdb00bc465c. 2020-11-17 18:32:31,328 WARN [RpcServer.default.FPBQ.Fifo.handler=10,queue=0,port=16201] ipc.RpcServer: (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1605637651246","responsesize":"2684","method":"Multi","param":"region\u003d aeris_v2,\\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259., for 2 action(s) and 1st row key\u003d\\x0B\\x06\\x90","processingtimems":300082,"client":"192.168.2.21:39536","queuetimems":974522,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":2} 2020-11-17 18:32:31,330 ERROR [RS_CLOSE_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Memstore data size is 42718364 2020-11-17 18:32:31,330 INFO [RS_CLOSE_REGION-regionserver/cvp21:16201-0] regionserver.HRegion: Closed aeris_v2,\x0B,1598623754299.06f7ca9534710e37d90d9f8345864259. 2020-11-17 18:32:31,345 INFO [regionserver/cvp21:16201] regionserver.HRegionServer: stopping server cvp21,16201,1605569374704; all regions closed. 2020-11-17 18:37:31,268 INFO [regionserver/cvp21:16201.Chore.4] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 30012 ns. 2020-11-17 18:42:31,337 INFO [regionserver/cvp21:16201.Chore.4] hbase.ScheduledChore: CompactionThroughputTuner average execution time: 20582 ns. 2020-11-17 18:47:31,364 WARN [Abort regionserver monitor] regionserver.HRegionServer: Aborting region server timed out, terminating forcibly and does not wait for any running shutdown hooks or finalizers to finish their work. Thread dump to stdout.